Overview

Monitoring API is an API for monitoring news for different search keywords. It uses the same standard interaction procedure, as the Search API (see Introduction section). The base url for the API is api.datanews.io/v1/monitors.

Monitor

The API operates with objects called monitors. The monitor is an object that stores the data about what keywords you would like to monitor and how you want to receive the results. At the moment, you can receive the results via the API or a webhook, receiving via an email is on the roadmap. Here is an example monitor object:

Monitor 
{
    "id": "d2f8d80bd6154939880fd96234ee0656e",
    "query": "Starship",
    "webhook": "https://mywebhook.com",
    "active": true,
    "last_run_id": "2ab634b3bf8d4dfgb04f99fbf0d524ef",
    "last_run_time": "2020-08-09T11:49:08.702318"
}

It says that we would like to search for articles that contain Starship and want them to be sent to https://mywebhook.com.

Additional fields of the Monitor:

  • active - shows the status of the monitor, disabled (false) or running regularly (true)
  • last_run_id - shows the run_id of the last run, initial starting point for retrieving the articles
  • last_run_time - shows the time in iso format, when the monitor was last ran

Run

The monitors are being run with a time interval between them. When being run, the monitor collects all the new articles relevant to the search phrase and then sends them to the provided webhook. The result articles are also stored in the database, allowing to be retrieved via the API (namely using the /monitors/latest endpoint).

Each run is identified by the run_id. It contains the following fields:

  • id - unique identifier of the run
  • monitor_id - unique identifier of the monitor that the run belongs to
  • query - query that had been run
  • run_time - time that the monitor had been run in ISO format
  • last_run_id - id of the previous run
  • last_run_time - time of the previous run in ISO format
  • articles - articles that were found during the run

The Run object contains the last_run_id, which is the id of the previous run. This link between runs allows for all of the runs to be retrieved sequentially, one by one. Note, that when the query returns too many results, the run may be split into several run objects with same run_time field.

Run 
{
   "id": "85219cd9652e4f02a22d92aa23c15675",
   "monitor_id": "dfc22b6dc636494e8a02b58250eda2b5",
   "query": "Netflix",
   "run_time": "2020-09-14T10:07:56.685901",
   "last_run_id": null,
   "last_run_time": null,
   "articles": [
      {
         "url": "https://junkee.com/cuties-netflix-contoversy-cancelnetflix-explainer/270619",
         "source": "junkee.com",
         "authors": [],
         "title": "How Netflix's 'Cuties' Became The Most Misunderstood Film Of The Year - Junkee",
         "pubDate": "2020-09-14T07:24:56+00:00",
         "country": "",
         "language": "en",
         "description": "How Netflix's 'Cuties' Became The Most Misunderstood Film Of The Year  Junkee",
         "imageUrl": "https://junkee.com/wp-content/uploads/2020/09/cuties-netflix.jpg",
         "content": "In February, 'Cuties' was a film festival favourite. Now, 600,000 people have signed a petition demanding everyone involved be charged with child exploitation.\n\nCuties, a French film acquired and dubbed into 40+ languages by Netflix after it received ... [+7501 chars]"
      },
      ...
   ]
}

Endpoints

The API consists of the following endpoints:

  • POST api.datanews.io/v1/monitors/create - create new monitor
  • GET api.datanews.io/v1/monitors/list - list all monitors
  • GET api.datanews.io/v1/monitors/list/{id} - retrieve monitor by id
  • DELETE api.datanews.io/v1/monitors/delete/{id} - delete monitor by id
  • GET api.datanews.io/v1/monitors/latest/{run_id} - retrieve monitor run by run_id

See more detailed description of the endpoints on individual doc pages (see navigation).

Client Library

For your convenience we provide a client library for Python. Libraries for more languages are coming soon.

You can install the library with pip install datanews command in the shell.

An example usage of the library is shown below:

CLIENT LIBRARY 
import datanews
datanews.api_key = 'API_KEY'

# create new monitor, pass query (list of keywords), and webhook url
monitor = datanews.Monitor.create(query='SpaceX', 'webhook'='https://mywebhook.com')

# list all monitors
print(datanews.Monitor.list())

# get id of the monitor
id = monitor['id']

# list latest articles from the Monitor
articles = []
run_id = monitor.get('last_run_id', None)
done = run_id is None
while not done:
    run = datanews.Monitor.latest(run_id)
    articles.extend(run['articles'])
    run_id = run['last_run_id']
    done = run_id is None

print(len(articles))
Jump to