Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
S
serviceRec
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
郭羽
serviceRec
Commits
ffab6059
Commit
ffab6059
authored
3 years ago
by
郭羽
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
update feature
parent
ca0f2dde
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
4 additions
and
4 deletions
+4
-4
featureEng.py
spark/featureEng.py
+4
-4
No files found.
spark/featureEng.py
View file @
ffab6059
...
...
@@ -339,8 +339,8 @@ def featuresToRedis(samples,columns,prefix,redisKey):
k
=
d
[
idCol
]
v
=
json
.
dumps
(
d
.
asDict
(),
ensure_ascii
=
False
)
newKey
=
redisKey
+
k
#
conn.set(newKey, v)
#
conn.expire(newKey, 60 * 60 * 24 * 7)
conn
.
set
(
newKey
,
v
)
conn
.
expire
(
newKey
,
60
*
60
*
24
*
7
)
#根据timestamp获取每个user最新的记录
prefixSamples
=
samples
.
groupBy
(
idCol
)
.
agg
(
F
.
max
(
"timestamp"
)
.
alias
(
timestampCol
))
...
...
@@ -813,13 +813,13 @@ if __name__ == '__main__':
# model columns
print
(
"model columns to redis..."
)
model_columns
=
user_columns
+
item_columns
#
featureColumnsToRedis(model_columns)
featureColumnsToRedis
(
model_columns
)
print
(
"数据字典save..."
)
print
(
"dataVocab:"
,
str
(
dataVocab
.
keys
()))
vocab_path
=
"../vocab/{}_vocab.json"
.
format
(
VERSION
)
dataVocabStr
=
json
.
dumps
(
dataVocab
,
ensure_ascii
=
False
)
#
open(configUtils.VOCAB_PATH, mode='w', encoding='utf-8').write(dataVocabStr)
open
(
configUtils
.
VOCAB_PATH
,
mode
=
'w'
,
encoding
=
'utf-8'
)
.
write
(
dataVocabStr
)
"""特征数据存入redis======================================"""
# user特征数据存入redis
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment