Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
C
crawler
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
Chengyang Zhong
crawler
Commits
8c715c04
Commit
8c715c04
authored
4 years ago
by
litaolemo
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
update
parent
f4585e73
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
2 additions
and
2 deletions
+2
-2
cal_ni_and_put_to_backend.py
crawler_sys/scheduler/cal_ni_and_put_to_backend.py
+2
-2
No files found.
crawler_sys/scheduler/cal_ni_and_put_to_backend.py
View file @
8c715c04
...
@@ -252,7 +252,6 @@ def push_data_to_user(res_data: Dict) -> Dict:
...
@@ -252,7 +252,6 @@ def push_data_to_user(res_data: Dict) -> Dict:
:return:
:return:
"""
"""
qiniu_img_list
=
[]
qiniu_img_list
=
[]
content
=
""
if
res_data
[
"img_list"
]:
if
res_data
[
"img_list"
]:
for
img_url
in
res_data
[
"img_list"
]:
for
img_url
in
res_data
[
"img_list"
]:
try
:
try
:
...
@@ -270,6 +269,7 @@ def push_data_to_user(res_data: Dict) -> Dict:
...
@@ -270,6 +269,7 @@ def push_data_to_user(res_data: Dict) -> Dict:
res_data
[
"qiniu_img_list"
]
=
qiniu_img_list
res_data
[
"qiniu_img_list"
]
=
qiniu_img_list
if
"http://t.cn/"
in
res_data
[
"title"
]:
if
"http://t.cn/"
in
res_data
[
"title"
]:
res_data
[
"title"
]
=
res_data
[
"title"
]
.
split
(
"http://t.cn/"
)[
0
]
res_data
[
"title"
]
=
res_data
[
"title"
]
.
split
(
"http://t.cn/"
)[
0
]
res_data
[
"content"
]
=
res_data
[
"title"
]
elif
res_data
[
"platform"
]
==
"douban"
:
elif
res_data
[
"platform"
]
==
"douban"
:
content
=
res_data
.
get
(
"content"
)
content
=
res_data
.
get
(
"content"
)
if
content
:
if
content
:
...
@@ -391,7 +391,7 @@ def task_main():
...
@@ -391,7 +391,7 @@ def task_main():
for
res_data
in
scan_from_redis
(
rules_list
):
for
res_data
in
scan_from_redis
(
rules_list
):
tractate_id
=
write_data_into_mysql
(
res_data
)
tractate_id
=
write_data_into_mysql
(
res_data
)
try
:
try
:
print
(
res_data
)
print
(
"line394"
,
res_data
)
if
res_data
[
"level"
]
>=
1
and
tractate_id
:
if
res_data
[
"level"
]
>=
1
and
tractate_id
:
title_str
=
res_data
[
"platform"
]
+
"帖子内容审核"
title_str
=
res_data
[
"platform"
]
+
"帖子内容审核"
body_str
=
"""
body_str
=
"""
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment