Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
M
meta_base_code
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
宋柯
meta_base_code
Commits
df25d292
Commit
df25d292
authored
Nov 17, 2020
by
litaolemo
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
update
parent
74fa4f4d
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
8 additions
and
6 deletions
+8
-6
spark_test.py
task/spark_test.py
+8
-6
No files found.
task/spark_test.py
View file @
df25d292
...
@@ -93,13 +93,15 @@ spark.sql("CREATE TEMPORARY FUNCTION arrayMerge AS 'com.gmei.hive.common.udf.UDF
...
@@ -93,13 +93,15 @@ spark.sql("CREATE TEMPORARY FUNCTION arrayMerge AS 'com.gmei.hive.common.udf.UDF
# huidu_device_id_df = spark.sql(huidu_device_id_sql)
# huidu_device_id_df = spark.sql(huidu_device_id_sql)
# huidu_device_id_df.createOrReplaceTempView("dev_view")
# huidu_device_id_df.createOrReplaceTempView("dev_view")
sql_search_ctr
=
r"""
sql_search_ctr
=
r"""
SELECT *
SELECT *
FROM online.bl_hdfs_maidian_updates
FROM online.bl_hdfs_maidian_updates
WHERE partition_date >= '{start_date}'
WHERE partition_date >= '${start_date}'
AND partition_date < '{end_date}'
AND partition_date < '${end_date}'
AND action = 'on_click_navbar_search'
AND action = 'on_click_card'
limit 20
AND params['page_name']='welfare_home'
"""
.
format
(
start_date
=
'20201105'
,
end_date
=
'20201025'
)
AND params['card_type'] ='search_word'
AND params['in_page_pos']='大家都在搜'
"""
.
format
(
start_date
=
'20201116'
,
end_date
=
'20201117'
)
print
(
sql_search_ctr
)
print
(
sql_search_ctr
)
search_ctr_df
=
spark
.
sql
(
sql_search_ctr
)
search_ctr_df
=
spark
.
sql
(
sql_search_ctr
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment