Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
F
ffm-baseline
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ML
ffm-baseline
Commits
816cd8dd
Commit
816cd8dd
authored
Nov 13, 2018
by
王志伟
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
save to table(jerry_prod)
parent
e7b38822
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
6 additions
and
5 deletions
+6
-5
temp_analysis.scala
eda/feededa/src/main/scala/com/gmei/temp_analysis.scala
+6
-5
No files found.
eda/feededa/src/main/scala/com/gmei/temp_analysis.scala
View file @
816cd8dd
...
@@ -54,9 +54,9 @@ object temp_analysis {
...
@@ -54,9 +54,9 @@ object temp_analysis {
import
sc.implicits._
import
sc.implicits._
//
val stat_date = GmeiConfig.getMinusNDate(1)
val
stat_date
=
GmeiConfig
.
getMinusNDate
(
1
)
//println(param.date)
//println(param.date)
val
partition_date
=
param
.
date
.
replace
(
"-"
,
""
)
val
partition_date
=
stat_
date
.
replace
(
"-"
,
""
)
//获取策略命中用户device_id
//获取策略命中用户device_id
val
device_id_cover
=
sc
.
sql
(
val
device_id_cover
=
sc
.
sql
(
...
@@ -70,7 +70,7 @@ object temp_analysis {
...
@@ -70,7 +70,7 @@ object temp_analysis {
val
diary_pv
=
sc
.
sql
(
val
diary_pv
=
sc
.
sql
(
s
"""
s
"""
|select '${
param.
date}' as stat_date,count(params["business_id"]) as diary_pv,count(distinct(cl_id)) as device_num_diary
|select '${
stat_
date}' as stat_date,count(params["business_id"]) as diary_pv,count(distinct(cl_id)) as device_num_diary
|from online.tl_hdfs_maidian_view
|from online.tl_hdfs_maidian_view
|where action="page_view"
|where action="page_view"
|and params["page_name"]="diary_detail"
|and params["page_name"]="diary_detail"
...
@@ -81,7 +81,7 @@ object temp_analysis {
...
@@ -81,7 +81,7 @@ object temp_analysis {
val
meigou_pv
=
sc
.
sql
(
val
meigou_pv
=
sc
.
sql
(
s
"""
s
"""
|select '${
param.
date}' as stat_date,count(params["business_id"]) as meigou_pv,count(distinct(cl_id)) as device_num_meigou
|select '${
stat_
date}' as stat_date,count(params["business_id"]) as meigou_pv,count(distinct(cl_id)) as device_num_meigou
|from online.tl_hdfs_maidian_view
|from online.tl_hdfs_maidian_view
|where action="page_view"
|where action="page_view"
|and params["page_name"]="welfare_detail"
|and params["page_name"]="welfare_detail"
...
@@ -93,8 +93,9 @@ object temp_analysis {
...
@@ -93,8 +93,9 @@ object temp_analysis {
val
result
=
diary_pv
.
join
(
meigou_pv
,
"stat_date"
)
val
result
=
diary_pv
.
join
(
meigou_pv
,
"stat_date"
)
result
.
show
()
result
.
show
()
GmeiConfig
.
writeToJDBCTable
(
result
,
"diary_pv"
,
SaveMode
.
Append
)
result
.
select
(
"stat_date"
,
"diary_pv"
,
"device_num_diary"
,
"meigou_pv"
,
"device_num_meigou"
).
write
.
mode
(
SaveMode
.
Append
).
save
(
"/data2/test.txt"
)
//
result.select("stat_date","diary_pv","device_num_diary","meigou_pv","device_num_meigou").write.mode(SaveMode.Append).save("/data2/test.txt")
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment