Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
F
ffm-baseline
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ML
ffm-baseline
Commits
c3dcbe62
Commit
c3dcbe62
authored
May 06, 2019
by
王志伟
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
更改时间
parent
b1b98462
Show whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
2 deletions
+7
-2
data_feed_exposure_precise.scala
.../src/main/scala/com/gmei/data_feed_exposure_precise.scala
+7
-2
No files found.
eda/feededa/src/main/scala/com/gmei/data_feed_exposure_precise.scala
View file @
c3dcbe62
...
...
@@ -8,9 +8,9 @@ import org.apache.spark.sql.SaveMode
import
org.apache.log4j.
{
Level
,
Logger
}
import
scopt.OptionParser
import
com.gmei.lib.AbstractParams
import
org.apache.spark.sql.expressions.Window
import
org.apache.spark.sql.functions.row_number
import
org.apache.spark.sql.types.DoubleType
object
data_feed_exposure_precise
{
...
...
@@ -762,8 +762,13 @@ object tag_value {
val
tag_level_index
=
result
.
rdd
.
map
(
row
=>(
row
(
0
).
toString
,
row
(
1
).
toString
)).
map
(
row
=>(
row
.
_1
.
split
(
","
).
head
,
row
.
_1
.
split
(
","
)(
1
),
row
.
_2
)).
toDF
(
"tag_id"
,
"level_id"
,
"index"
)
// val colNames = tag_level_index.columns
// val cols = colNames.map(f => f.toInt)
//
val
test
=
tag_level_index
.
select
(
tag_level_index
.
col
(
"tag_id"
).
cast
(
DoubleType
).
as
(
"tag_id"
),
tag_level_index
.
col
(
"level_id"
).
cast
(
DoubleType
).
as
(
"level_id"
),
tag_level_index
.
col
(
"index"
).
cast
(
DoubleType
).
as
(
"index"
))
println
(
"开始写入"
)
GmeiConfig
.
writeToJDBCTable
(
"jdbc:mysql://172.16.40.158:4000/jerry_prod?user=root&password=3SYz54LS9#^9sBvC&rewriteBatchedStatements=true"
,
t
ag_level_index
,
table
=
"tag_level_index"
,
SaveMode
.
Overwrite
)
GmeiConfig
.
writeToJDBCTable
(
"jdbc:mysql://172.16.40.158:4000/jerry_prod?user=root&password=3SYz54LS9#^9sBvC&rewriteBatchedStatements=true"
,
t
est
,
table
=
"tag_level_index"
,
SaveMode
.
Overwrite
)
println
(
"写入完成"
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment