create a python script that will monitor the object storage
I have a small project for you let me know if you want to work on it
It is a simple task so need to be completed in 1 day
One of my job drops files to an object storage and the name of the files is in the format of yyyymmddTABLE1 , yyyymmddTABLE2 …….
Now I want to monitor my job if it really dropped the files to the storage on that specific date
So one way which I think of is
Create a python job
1) that will query the object storage using the CLI
oci os object list -bn abc-mno-bucket -ns bbcmm --auth instance_principal
[login to view URL]
2) grab the output which will be in the form of
{
"data": [
{
"md5": "55sYrq31fJ1VgHfEGIh2hg==",
"name": " yyyymmddTABLE1 ",
"size": 1785862,
"time-created": "2019-08-13T23:31:02.981000+00:00"
},
{
"md5": "uzobT/eOYWpieAdIu1lc2w==",
"name": " yyyymmddTABLE12",
"size": 466,
"time-created": "2019-08-13T20:09:54.148000+00:00"
},
],
"prefixes": []
}
3) validate any one table say yyyymmddTABLE1 if the time_created = get date ()
get date = job execution date
4) if true push the exit code 0
5) if not push the exit code 1
I am working as a Data Scientist with
Python, TensorFlow Framework, Github, Scrapping, Analysis and Modeling
Matlab Analysis and Modeling, Predictive Modelling, and Statistical Modeling using Machine Learning and Deep Learning Algorithms in these fields.