Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
M
meta_base_code
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
黎涛
meta_base_code
Commits
cb7e30b0
Commit
cb7e30b0
authored
Nov 26, 2020
by
litaolemo
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
update
parent
2f80d490
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
25 additions
and
2 deletions
+25
-2
daily_search_word_count_last_two_year.py
task/daily_search_word_count_last_two_year.py
+25
-2
No files found.
task/daily_search_word_count_last_two_year.py
View file @
cb7e30b0
...
...
@@ -122,14 +122,37 @@ spark.sql("CREATE TEMPORARY FUNCTION json_map AS 'brickhouse.udf.json.JsonMapUDF
spark
.
sql
(
"CREATE TEMPORARY FUNCTION is_json AS 'com.gmei.hive.common.udf.UDFJsonFormatCheck'"
)
spark
.
sql
(
"CREATE TEMPORARY FUNCTION arrayMerge AS 'com.gmei.hive.common.udf.UDFArryMerge'"
)
tableDF
=
spark
.
read
.
format
(
"jdbc"
)
.
\
tableDF
1
=
spark
.
read
.
format
(
"jdbc"
)
.
\
option
(
"url"
,
"jdbc:mysql://172.16.30.143/zhengxing"
)
.
\
option
(
"driver"
,
"com.mysql.jdbc.Driver"
)
.
\
option
(
"dbtable"
,
"wiki_item"
)
.
\
option
(
"user"
,
"work"
)
.
\
option
(
"password"
,
"BJQaT9VzDcuPBqkd"
)
.
load
()
tableDF
.
show
()
tableDF
1
.
show
()
tableDF2
=
spark
.
read
.
format
(
"jdbc"
)
.
\
option
(
"url"
,
"jdbc:mysql://172.16.30.143/zhengxing"
)
.
\
option
(
"driver"
,
"com.mysql.jdbc.Driver"
)
.
\
option
(
"dbtable"
,
"wiki_product"
)
.
\
option
(
"user"
,
"work"
)
.
\
option
(
"password"
,
"BJQaT9VzDcuPBqkd"
)
.
load
()
tableDF2
.
show
()
tableDF3
=
spark
.
read
.
format
(
"jdbc"
)
.
\
option
(
"url"
,
"jdbc:mysql://172.16.30.143/zhengxing"
)
.
\
option
(
"driver"
,
"com.mysql.jdbc.Driver"
)
.
\
option
(
"dbtable"
,
"wiki_collect"
)
.
\
option
(
"user"
,
"work"
)
.
\
option
(
"password"
,
"BJQaT9VzDcuPBqkd"
)
.
load
()
tableDF3
.
show
()
tableDF4
=
spark
.
read
.
format
(
"jdbc"
)
.
\
option
(
"url"
,
"jdbc:mysql://172.16.30.143/zhengxing"
)
.
\
option
(
"driver"
,
"com.mysql.jdbc.Driver"
)
.
\
option
(
"dbtable"
,
"wiki_brand"
)
.
\
option
(
"user"
,
"work"
)
.
\
option
(
"password"
,
"BJQaT9VzDcuPBqkd"
)
.
load
()
tableDF4
.
show
()
# df = spark.read.jdbc(url="jdbc:mysql://172.16.30.143/zhengxing?user=work&password=BJQaT9VzDcuPBqkd&rewriteBatchedStatements=true",table="wiki_item")
# print(huidu_device_id_sql)
# huidu_device_id_df = spark.sql(huidu_device_id_sql)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment