Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
F
ffm-baseline
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ML
ffm-baseline
Commits
b9f39289
Commit
b9f39289
authored
Oct 09, 2018
by
高雅喆
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
add getOrElse
parent
d9b4c46d
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
6 deletions
+2
-6
Main.scala
eda/node2vec/src/main/scala/com/gmei/Main.scala
+2
-6
No files found.
eda/node2vec/src/main/scala/com/gmei/Main.scala
View file @
b9f39289
...
...
@@ -184,7 +184,6 @@ object Main {
GmeiConfig
.
writeToJDBCTable
(
similar_result
,
table
=
"nd_cid_similarity_matrix"
,
SaveMode
.
Overwrite
)
//3. cids queue map to device_id
ti
.
tidbMapTable
(
dbName
=
GmeiConfig
.
config
.
getString
(
"tidb.database"
),
tableName
=
"nd_cid_similarity_matrix"
)
...
...
@@ -198,14 +197,14 @@ object Main {
|on a.cid = b.cid
|where b.similarity_cid is not null
"""
.
stripMargin
).
na
.
fill
(
Map
(
"city_id"
->
"beijing"
))
).
na
.
fill
(
Map
(
"city_id"
->
"beijing"
))
device_id
.
show
()
val
device_queue
=
device_id
.
rdd
.
map
{
item
=>
val
parts
=
(
item
.
getAs
[
String
](
fieldName
=
"device_id"
),
item
.
getAs
[
String
](
fieldName
=
"city_id"
),
item
.
getAs
[
String
](
fieldName
=
"similarity_cid"
))
Try
{
(
parts
.
_1
,
Try
(
parts
.
_2
.
toString
.
replace
(
"worldwide"
,
"beijing"
)),
Try
(
parts
.
_3
.
toString
.
replace
(
"diary|"
,
""
)).
getOrElse
(
null
))
(
parts
.
_1
,
Try
(
parts
.
_2
.
toString
.
replace
(
"worldwide"
,
"beijing"
))
.
getOrElse
(
null
)
,
Try
(
parts
.
_3
.
toString
.
replace
(
"diary|"
,
""
)).
getOrElse
(
null
))
}.
getOrElse
(
null
)
}.
filter
(
_
!=
null
).
toDF
(
"device_id"
,
"city_id"
,
"similarity_cid"
)
...
...
@@ -220,6 +219,3 @@ object Main {
sys
.
exit
(
1
)
}
}
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment