Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
F
ffm-baseline
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ML
ffm-baseline
Commits
6bcae79b
Commit
6bcae79b
authored
Oct 31, 2018
by
张彦钊
Browse files
Options
Browse Files
Download
Plain Diff
Merge branch 'master' of git.wanmeizhensuo.com:ML/ffm-baseline
不对机构黑名单做处理
parents
8feb3cde
c530ff4b
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
33 additions
and
5 deletions
+33
-5
WeafareStat.scala
eda/feededa/src/main/scala/com/gmei/WeafareStat.scala
+33
-5
No files found.
eda/feededa/src/main/scala/com/gmei/WeafareStat.scala
View file @
6bcae79b
...
@@ -210,8 +210,8 @@ object NdDataInput {
...
@@ -210,8 +210,8 @@ object NdDataInput {
ti
.
tidbMapTable
(
dbName
=
"jerry_prod"
,
tableName
=
"data_feed_click"
)
ti
.
tidbMapTable
(
dbName
=
"jerry_prod"
,
tableName
=
"data_feed_click"
)
ti
.
tidbMapTable
(
dbName
=
"eagle"
,
tableName
=
"feed_diary_boost"
)
ti
.
tidbMapTable
(
dbName
=
"eagle"
,
tableName
=
"feed_diary_boost"
)
val
date8
=
GmeiConfig
.
getMinusNDate
(
5
)
val
date8
=
GmeiConfig
.
getMinusNDate
(
70
)
val
tidb_inupt
=
sc
.
sql
(
val
result00
=
sc
.
sql
(
s
"""
s
"""
|SELECT
|SELECT
| split(service_id,'\\\\|')[1] as sid,split(cid,'\\\\|')[1] as cid
| split(service_id,'\\\\|')[1] as sid,split(cid,'\\\\|')[1] as cid
...
@@ -219,9 +219,9 @@ object NdDataInput {
...
@@ -219,9 +219,9 @@ object NdDataInput {
|where stat_date > '${date8}'
|where stat_date > '${date8}'
"""
.
stripMargin
"""
.
stripMargin
)
)
tidb_inupt
.
createOrReplaceTempView
(
"tmp1"
)
result00
.
createOrReplaceTempView
(
"tmp1"
)
tidb_inupt
.
show
()
result00
.
show
()
println
(
tidb_inupt
.
count
())
println
(
result00
.
count
())
val
yesteday
=
GmeiConfig
.
getMinusNDate
(
1
).
replace
(
"-"
,
""
)
val
yesteday
=
GmeiConfig
.
getMinusNDate
(
1
).
replace
(
"-"
,
""
)
val
result01
=
sc
.
sql
(
val
result01
=
sc
.
sql
(
...
@@ -238,6 +238,34 @@ object NdDataInput {
...
@@ -238,6 +238,34 @@ object NdDataInput {
result01
.
show
()
result01
.
show
()
println
(
result01
.
count
())
println
(
result01
.
count
())
val
result02
=
sc
.
sql
(
s
"""
|select a.sid as sid, a.cid as cid, a.ctag_id as ctag_id, a.clevel1_id as clevel1_id,
| b.tag_id as stag_id, c.level1_id as slevel1_id
|from tmp2 a
|left join online.tl_meigou_servicetag_view b on a.sid=b.service_id
|left join online.bl_tag_hierarchy_detail c on b.tag_id=c.id
|where b.partition_date='${yesteday}'
|and c.partition_date='${yesteday}'
"""
.
stripMargin
)
result02
.
createOrReplaceTempView
(
"tmp3"
)
result02
.
show
()
println
(
result02
.
count
())
val
tidb_input
=
sc
.
sql
(
s
"""
|select sid as service_id,cid
|from tmp3
|where clevel1_id = slevel1_id
"""
.
stripMargin
)
tidb_input
.
show
()
println
(
tidb_input
.
count
())
}
}
}
}
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment