Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
F
ffm-baseline
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ML
ffm-baseline
Commits
42ab11d1
Commit
42ab11d1
authored
Aug 16, 2018
by
张彦钊
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
modify multiFFMFormatPandas
parent
6cb6342a
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
12 deletions
+2
-12
utils.py
utils.py
+2
-12
No files found.
utils.py
View file @
42ab11d1
...
...
@@ -120,20 +120,11 @@ class multiFFMFormatPandas:
# 设置进程的数量
pool
=
Pool
(
processes
)
print
(
"总进度: "
+
str
(
len
(
data_list
)))
result_map
=
{}
for
i
in
range
(
len
(
data_list
)):
data_list
[
i
]
=
pool
.
apply_async
(
self
.
pool_function
,
(
data_list
[
i
],
t
,))
result_map
.
update
(
data_list
[
i
]
.
get
())
result_map
=
{}
for
i
in
data_list
:
result_map
.
update
(
i
.
get
())
'''
# 使用生成器方法计算,配合data_split_line的一起使用
result_map = {}
for i in data_list:
s = pool.apply_async(self.pool_function, (i, t,))
result_map.update(s.get())
'''
pool
.
close
()
pool
.
join
()
...
...
@@ -141,7 +132,6 @@ class multiFFMFormatPandas:
# 多进程计算方法
def
pool_function
(
self
,
df
,
t
):
s
=
{
idx
:
self
.
transform_row_
(
row
,
t
)
for
idx
,
row
in
df
.
iterrows
()}
return
{
idx
:
self
.
transform_row_
(
row
,
t
)
for
idx
,
row
in
df
.
iterrows
()}
# 切分数据方法,传人dataframe和切分条数的步长,返回dataframe的集合,每个dataframe中含有若干条数据
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment