Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
F
ffm-baseline
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ML
ffm-baseline
Commits
1103b210
Commit
1103b210
authored
Jul 02, 2019
by
张彦钊
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
修改esmm 写入数据表,用来测试
parent
ecafcb00
Show whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
2 additions
and
1 deletion
+2
-1
feature_engineering.py
eda/esmm/Model_pipline/feature_engineering.py
+1
-0
train.py
eda/esmm/Model_pipline/train.py
+1
-1
No files found.
eda/esmm/Model_pipline/feature_engineering.py
View file @
1103b210
...
@@ -373,6 +373,7 @@ if __name__ == '__main__':
...
@@ -373,6 +373,7 @@ if __name__ == '__main__':
spark
=
SparkSession
.
builder
.
config
(
conf
=
sparkConf
)
.
enableHiveSupport
()
.
getOrCreate
()
spark
=
SparkSession
.
builder
.
config
(
conf
=
sparkConf
)
.
enableHiveSupport
()
.
getOrCreate
()
ti
=
pti
.
TiContext
(
spark
)
ti
=
pti
.
TiContext
(
spark
)
ti
.
tidbMapDatabase
(
"jerry_test"
)
ti
.
tidbMapDatabase
(
"jerry_test"
)
# ti.tidbMapDatabase("jerry_prod")
ti
.
tidbMapDatabase
(
"eagle"
)
ti
.
tidbMapDatabase
(
"eagle"
)
spark
.
sparkContext
.
setLogLevel
(
"WARN"
)
spark
.
sparkContext
.
setLogLevel
(
"WARN"
)
path
=
"hdfs:///strategy/esmm/"
path
=
"hdfs:///strategy/esmm/"
...
...
eda/esmm/Model_pipline/train.py
View file @
1103b210
...
@@ -400,7 +400,7 @@ def update_or_insert(df2,queue_name):
...
@@ -400,7 +400,7 @@ def update_or_insert(df2,queue_name):
cur
=
con
.
cursor
()
cur
=
con
.
cursor
()
try
:
try
:
for
i
in
range
(
0
,
device_count
):
for
i
in
range
(
0
,
device_count
):
query
=
"""INSERT INTO esmm_device_diary_queue (device_id, city_id, time,
%
s) VALUES('
%
s', '
%
s', '
%
s', '
%
s')
\
query
=
"""INSERT INTO esmm_device_diary_queue
_tmp
(device_id, city_id, time,
%
s) VALUES('
%
s', '
%
s', '
%
s', '
%
s')
\
ON DUPLICATE KEY UPDATE device_id='
%
s', city_id='
%
s', time='
%
s',
%
s='
%
s'"""
%
(
queue_name
,
df2
.
device_id
[
i
],
df2
.
city_id
[
i
],
df2
.
time
[
i
],
df2
[
queue_name
][
i
],
df2
.
device_id
[
i
],
df2
.
city_id
[
i
],
df2
.
time
[
i
],
queue_name
,
df2
[
queue_name
][
i
])
ON DUPLICATE KEY UPDATE device_id='
%
s', city_id='
%
s', time='
%
s',
%
s='
%
s'"""
%
(
queue_name
,
df2
.
device_id
[
i
],
df2
.
city_id
[
i
],
df2
.
time
[
i
],
df2
[
queue_name
][
i
],
df2
.
device_id
[
i
],
df2
.
city_id
[
i
],
df2
.
time
[
i
],
queue_name
,
df2
[
queue_name
][
i
])
cur
.
execute
(
query
)
cur
.
execute
(
query
)
con
.
commit
()
con
.
commit
()
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment