How to build news aggregator API with FastApi

In this article, we will be building an application that aggregates news from two different APIs. The APIs we’ll be using are Reddit and News API.
The two functionalities that we will be implementing are “list” and “search”, and we will be adding unit tests for these functionalities.
You will need to go to https://newsapi.org/ to get your Api-Key, while you don’t need any for the reddit requests.

This is an example request for a generic GET request which should fulfill the “list”.

> Request
GET /news   HTTP/1.1
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ
Accept: application/json
> Response
[
  {
    "headline": "Human organs can be stored for three times as long in major breakthrough for transplants",  // Headline of the article
    "link": "https://www.telegraph.co.uk/science/2019/09/09/human-organs-can-stored-three-times-long-major-breakthrough/",  // Link of the article
    "source": "reddit" // Source that you retrieved this news from
  },
  {
    "headline": "Depth of Field: The Shared Memory of One World Trade Center",
    "link": "https://www.wired.com/story/one-world-trade-center-history-future/",
    "source": "newsapi"
  },
]

For the “search”, this is an example

> Request
GET /news?query=bitcoin   HTTP/1.1
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ
Accept: application/json

> Response
[
  {
    "headline": "IRS goes after cryptocurrency owners for unpaid taxes",
    "link": "https://www.cbsnews.com/news/own-bitcoin-irs-pursues-cryptocurrency-owners-for-unpaid-taxes/",
    "source": "reddit"
  },
  {
    "headline": "Skirting US sanctions, Cubans flock to cryptocurrency to shop online, send funds",
    "link": "https://www.channelnewsasia.com/news/business/skirting-us-sanctions--cubans-flock-to-cryptocurrency-to-shop-online--send-funds-11901148",
    "source": "newsapi"
  },
]

LET'S GET STARTED
Go ahead and set up your virtualenv and activate it

python3 -m venv env
source env/bin/activate

Create a folder called “newaggregator” and the following inside the folder.
“main.py”, “requirements.txt”, “src”, and “tests”. (src and tests are folders while main.py and requirements.txt are files). At this point, your folder structure should be looking like this

├── src
└── tests
└── main.py
└── requirements.txt

Copy the following inside the requirements.txt file

python-dotenv==0.20.0
fastapi==0.75.1
uvicorn==0.15.0
responses==0.20.0
pytest==7.1.1

Run the following command to install the packages

pip install -r requirements.txt

Create a “.env” file and add the following

NEWS_API_KEY = 'your news api key'

With our packages installed, go ahead and create two more folders inside the “src” folder and name them “queries” and “utils”.
Inside “main.py” add the following.

Here we will be using on route “/news” to both list our news aggregate and search for a particular keyword. Our method will adjust based on the parameter passed. You will notice we imported a QueryManager but currently we don’t have it. So let’s go ahead and work on the “queries” folder.

Inside the “queries” folder, create a file called “run_queries.py” and add the following code.

We just created a “QueryManager” class and added two methods “search_news_query” and “get_news_query”.
“get_news_query” will be called when we want to return a list of news from Reddit and News_api.
“search_news_query” will be called when we want to search with a keyword for news containing that keyword.

Navigate to our “utils” folder and create the following files “exceptions.py”, “external_api_call.py” and “external_libs_config”.

Add the following code inside the “external_api_call.py”

Next, add the following inside “external_libs_config”

For the “exceptions.py”, add the following

With all that done, go ahead and start your local server using

uvicorn main:app --reload

Finally, navigate to “127.0.0.1:8000/docs” you will see the swagger-UI, test your endpoint there and you will see that it works.

Now let's add our test cases.
For this, we will be using pytest and responses library.
Inside your “tests” folder, create another folder called “queries” and add “test_run_queries.py” inside it.
Inside the “test_run_queries.py”, add the following code.

Go ahead and run pytest command to run the testcases and all testcases should be passing.

pytest tests

At this point, your folder structure should look like this

├── src
    └── queries
        └── run_queries.py
    └── utils
        └── exceptions.py
        └── external_api_calls.py
        └── external_libs_config.py
└── tests
    └── queries
        └── test_run_queries.py
└── main.py
└── requirements.txt
└── .env

With all that done, we have been able to build a new aggregator using FastApi utilizing Reddit and New_Api.