Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
F
ffm-baseline
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ML
ffm-baseline
Commits
a2049b7f
Commit
a2049b7f
authored
Apr 25, 2019
by
张彦钊
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
修改测试文件
parent
6122a37d
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
7 additions
and
5 deletions
+7
-5
multi.py
tensnsorflow/multi.py
+7
-5
No files found.
tensnsorflow/multi.py
View file @
a2049b7f
...
@@ -18,15 +18,15 @@ def app_list_func(x,l):
...
@@ -18,15 +18,15 @@ def app_list_func(x,l):
return
","
.
join
([
str
(
j
)
for
j
in
e
])
return
","
.
join
([
str
(
j
)
for
j
in
e
])
def
multi_hot
(
df
,
column
,
n
):
def
multi_hot
(
df
,
column
,
n
):
df
[
column
]
=
df
[
column
]
.
fillna
(
"lost_na"
)
df
=
df
.
select
[
column
]
.
fillna
(
"lost_na"
)
app_list_value
=
[
i
.
split
(
","
)
for
i
in
df
[
column
]
.
unique
()]
app_list_value
=
[
i
.
split
(
","
)
for
i
in
df
.
select
[
column
]
.
unique
()]
app_list_unique
=
[]
app_list_unique
=
[]
for
i
in
app_list_value
:
for
i
in
app_list_value
:
app_list_unique
.
extend
(
i
)
app_list_unique
.
extend
(
i
)
app_list_unique
=
list
(
set
(
app_list_unique
))
app_list_unique
=
list
(
set
(
app_list_unique
))
number
=
len
(
app_list_unique
)
number
=
len
(
app_list_unique
)
app_list_map
=
dict
(
zip
(
app_list_unique
,
list
(
range
(
n
,
number
+
n
))))
app_list_map
=
dict
(
zip
(
app_list_unique
,
list
(
range
(
n
,
number
+
n
))))
df
[
column
]
=
df
[
column
]
.
apply
(
app_list_func
,
args
=
(
app_list_map
,))
df
=
df
.
select
[
column
]
.
apply
(
app_list_func
,
args
=
(
app_list_map
,))
return
number
,
app_list_map
return
number
,
app_list_map
def
feature_engineer
():
def
feature_engineer
():
...
@@ -75,11 +75,13 @@ def feature_engineer():
...
@@ -75,11 +75,13 @@ def feature_engineer():
hospital
=
spark
.
sql
(
sql
)
hospital
=
spark
.
sql
(
sql
)
df
=
df
.
join
(
hospital
,
"diary_service_id"
,
"left_outer"
)
.
fillna
(
"na"
)
df
=
df
.
join
(
hospital
,
"diary_service_id"
,
"left_outer"
)
.
fillna
(
"na"
)
print
(
df
.
count
())
df
=
df
.
drop
(
"level2"
)
.
drop
(
"diary_service_id"
)
df
=
df
.
drop
(
"level2"
)
.
drop
(
"diary_service_id"
)
df
=
df
.
drop_duplicates
([
"ucity_id"
,
"level2_ids"
,
"ccity_name"
,
"device_type"
,
"manufacturer"
,
df
=
df
.
drop_duplicates
([
"ucity_id"
,
"level2_ids"
,
"ccity_name"
,
"device_type"
,
"manufacturer"
,
"channel"
,
"top"
,
"time"
,
"stat_date"
,
"app_list"
,
"hospital_id"
,
"level3_ids"
])
"channel"
,
"top"
,
"time"
,
"stat_date"
,
"app_list"
,
"hospital_id"
,
"level3_ids"
])
print
(
df
.
count
())
multi_hot
(
df
,
"app_list"
,
1
)
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment