Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
F
ffm-baseline
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
ML
ffm-baseline
Commits
26ec027b
Commit
26ec027b
authored
6 years ago
by
张彦钊
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
change test file
parent
36a45caf
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
19 additions
and
19 deletions
+19
-19
multi.py
tensnsorflow/multi.py
+19
-19
No files found.
tensnsorflow/multi.py
View file @
26ec027b
...
...
@@ -198,30 +198,30 @@ def con_sql(db,sql):
def
test
():
sql
=
"select stat_date,cid_id from esmm_train_data e where stat_date = '{}' limit 60"
.
format
(
"2019-04-25"
)
# sql = "select stat_date,cid_id from esmm_train_data e where stat_date = '{}' limit 60".format("2019-04-25")
#
# df = spark.sql(sql)
# df.show(6)
#
# # df.write.csv('/recommend/tr', mode='overwrite', header=True)
# df.write.format("avro").save(path="/recommend/tr", mode="overwrite")
df
=
spark
.
sql
(
sql
)
df
.
show
(
6
)
# df.write.csv('/recommend/tr', mode='overwrite', header=True)
df
.
write
.
format
(
"avro"
)
.
save
(
path
=
"/recommend/tr"
,
mode
=
"overwrite"
)
# df.write.format("avro").save("/recommend/tr/avro.csv")
#
# from hdfs import InsecureClient
# from hdfs.ext.dataframe import read_dataframe
# from hdfs.ext.dataframe import write_dataframe
#
#
# client = InsecureClient('http://nvwa01:50070')
# # write_dataframe(client, '/recommend/va/a.csv', df)
#
# df = read_dataframe(client,"/recommend/va/a.csv")
#
#
# print(df.head())
from
hdfs
import
InsecureClient
from
hdfs.ext.dataframe
import
read_dataframe
from
hdfs.ext.dataframe
import
write_dataframe
client
=
InsecureClient
(
'http://nvwa01:50070'
)
df
=
read_dataframe
(
client
,
"/recommend/tr/part-00000-80d4e128-4a79-41de-9473-e4d0c5665047-c000.avro"
)
print
(
df
.
head
())
# spark.sql("use online")
# spark.sql("ADD JAR /srv/apps/brickhouse-0.7.1-SNAPSHOT.jar")
...
...
This diff is collapsed.
Click to expand it.
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment