Skip to content
Projects
Groups
Snippets
Help
Loading...
Sign in
Toggle navigation
B
bi-report
Project
Project
Details
Activity
Cycle Analytics
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Charts
Issues
0
Issues
0
List
Board
Labels
Milestones
Merge Requests
0
Merge Requests
0
CI / CD
CI / CD
Pipelines
Jobs
Schedules
Charts
Wiki
Wiki
Snippets
Snippets
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Charts
Create a new issue
Jobs
Commits
Issue Boards
Open sidebar
data
bi-report
Commits
b364c0c6
Commit
b364c0c6
authored
Apr 24, 2020
by
赵建伟
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
update codes
parent
f2dc0bef
Show whitespace changes
Inline
Side-by-side
Showing
3 changed files
with
19 additions
and
17 deletions
+19
-17
demo.zip
jobs/demo/demo.zip
+0
-0
step2.job
jobs/demo/step2.job
+2
-2
sparksql2email.sh
lib/shell/sparksql2email.sh
+17
-15
No files found.
jobs/demo/demo.zip
0 → 100644
View file @
b364c0c6
File added
jobs/demo/step2.job
View file @
b364c0c6
#step2.job
type=command
dependencies=step1_1,step1_2,step1_3,step1_4,step1_5,step1_6
command=sh /srv/apps/bi-report/lib/shell/sparksql2email.sh /srv/apps/bi-report/sqls/demo_param.sql /data/bi-report/result/demo_param_`date +"%Y-%m-%d"`.csv zhaojianwei zhaojianwei
\ No newline at end of file
command=sh /srv/apps/bi-report/lib/shell/sparksql2email.sh demo zhaojianwei zhaojianwei
\ No newline at end of file
lib/shell/sparksql2email.sh
View file @
b364c0c6
...
...
@@ -3,13 +3,15 @@
source
~/.bashrc
source
/etc/profile
if
[
$#
-lt
4
]
;
then
if
[
$#
-lt
3
]
;
then
echo
"Param error, please check again!"
exit
0
fi
#默认取输入的账期
V_DATE
=
$5
V_DATE
=
$4
timestamp
=
`
date
+
"%Y%m%d%H%M%S"
`
tmp_file_name
=
"/data/bi-report/tmp/tmp_"
$timestamp
".sql"
#输入的账期为空,则取T+1默认账期
if
[
-z
${
V_DATE
}
]
;
then
...
...
@@ -41,25 +43,24 @@ echo "V_7DAY_M_D_AGO: "$V_7DAY_M_D_AGO
echo
"************************************"
echo
""
#cp sqls/$1".sql" tmp.sql
cp
$1
tmp.sql
sed
-i
"s/
\$
V_PARYMD/
$V_PARYMD
/g"
tmp.sql
sed
-i
"s/
\$
V_PARY_M_D/
$V_PARY_M_D
/g"
tmp.sql
sed
-i
"s/
\$
V_1DAYMD_AGO/
$V_1DAYMD_AGO
/g"
tmp.sql
sed
-i
"s/
\$
V_7DAYMD_AGO/
$V_7DAYMD_AGO
/g"
tmp.sql
sed
-i
"s/
\$
V_1DAY_M_D_AGO/
$V_1DAY_M_D_AGO
/g"
tmp.sql
sed
-i
"s/
\$
V_7DAY_M_D_AGO/
$V_7DAY_M_D_AGO
/g"
tmp.sql
cp
$1
$tmp_file_name
sed
-i
"s/
\$
V_PARYMD/
$V_PARYMD
/g"
$tmp_file_name
sed
-i
"s/
\$
V_PARY_M_D/
$V_PARY_M_D
/g"
$tmp_file_name
sed
-i
"s/
\$
V_1DAYMD_AGO/
$V_1DAYMD_AGO
/g"
$tmp_file_name
sed
-i
"s/
\$
V_7DAYMD_AGO/
$V_7DAYMD_AGO
/g"
$tmp_file_name
sed
-i
"s/
\$
V_1DAY_M_D_AGO/
$V_1DAY_M_D_AGO
/g"
$tmp_file_name
sed
-i
"s/
\$
V_7DAY_M_D_AGO/
$V_7DAY_M_D_AGO
/g"
$tmp_file_name
# 新标签与新标签对应集合类型 的对应关系
/opt/spark/spark-2.4.5-bin-hadoop2.7/bin/beeline
-u
jdbc:hive2://bj-gm-prod-cos-datacenter003:10010/online
-n
data
--outputformat
=
csv2
--showHeader
=
true
-e
"
`
cat
tmp.sql
`
"
>
$2
rm
-rf
tmp.sql
/opt/spark/spark-2.4.5-bin-hadoop2.7/bin/beeline
-u
jdbc:hive2://bj-gm-prod-cos-datacenter003:10010/online
-n
data
--outputformat
=
csv2
--showHeader
=
true
-e
"
`
cat
$tmp_file_name
`
"
>
"/data/bi-report/result/"
$2
".csv"
rm
-rf
$tmp_file_name
to
=
$
3
cc
=
$
4
to
=
$
2
cc
=
$
3
echo
"to: "
$to
echo
"cc: "
$cc
python2.7 /srv/apps/bi-report/lib/python/sendmail.py
-t
$to
-s
自动发送
-m
$cc
-a
$2
python2.7 /srv/apps/bi-report/lib/python/sendmail.py
-t
$to
-s
自动发送
-m
$cc
-a
"/data/bi-report/result/"
$2
".csv"
exit
0
\ No newline at end of file
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment