Boris Chow   •   over 7 years ago

Setup schedule job for Sparkathon

Hi all, how can I retrieve & insert weather data periodically to the db / storage space ? I have already written a python script to retrieve weather data from IBM Weather Inside Service but I am not sure how to set up the scheduler job at Spark service.

• 1 comment
• 

  Serena Pietruszka Manager   •   over 7 years ago

  Hi Boris,

  Thanks for your question.

  Unfortunately, there is currently no way to schedule jobs. Notebooks are not designed to work that way as they terminate the session once the page is closed or after a period of inactivity. This feature is one that's in the works (via Spark-Submit) and is set to be available soon, but it may not be in time for the Sparkathon competition.

  Hope this helps!
  Serena

Comments are closed.