diff --git a/python/introduction/part-4.http/README.md b/python/introduction/part-4.http/README.md index f8359b8..706363b 100644 --- a/python/introduction/part-4.http/README.md +++ b/python/introduction/part-4.http/README.md @@ -3,10 +3,14 @@ So far, we've learnt Python fundamentals, worked with data, files and more importantly, basic data structures like `csv` and `json`. -JSON is a very popular format for now only storing, but transporting data in web HTTP applications.
+JSON is a very popular format for now only storing, but transporting data in web HTTP applications. Files are important because we may +need to read configuration files for our application.
One application could make a web request to ask for customer data from our application. By sending back JSON, the receiving application can easily process the data it gets. +Be sure to checkout [Part 2: Files](../part-2.files)
+As well as [Part 3: JSON](../part-3.json)
+ ## Python Dev Environment The same as Part 1, we start with a [dockerfile](./dockerfile) where we declare our version of `python`. @@ -32,7 +36,9 @@ Python 3.9.6 ## Our application -Firstly we have to import our dependencies +We're going to use what we've learnt in part 1,2 & 3 and create +our customer app that stores customer data
+Firstly we have to import our dependencies: ``` import os.path @@ -79,153 +85,328 @@ def updateCustomers(customers): customerFile.write(customerJSON) ``` -Now to recap and test our functions we need to define a list of customers -to use: +In the previous episode, we've created a `json` file to hold all our customers.
+We've learnt how to read and write to file and temporarily use the file for our storage.
``` -customers = { - "a": Customer("a","James", "Baker"), - "b": Customer("b", "Jonathan", "D"), - "c": Customer("c", "Aleem", "Janmohamed"), - "d": Customer("d", "Ivo", "Galic"), - "e": Customer("e", "Joel", "Griffiths"), - "f": Customer("f", "Michael", "Spinks"), - "g": Customer("g", "Victor", "Savkov"), - "h" : Customer("h", "Marcel", "Dempers") +{ + "a": { + "customerID": "a", + "firstName": "James", + "lastName": "Baker" + }, + "b": { + "customerID": "b", + "firstName": "Jonathan", + "lastName": "D" + }, + "c": { + "customerID": "c", + "firstName": "Aleem", + "lastName": "Janmohamed" + }, + "d": { + "customerID": "d", + "firstName": "Ivo", + "lastName": "Galic" + }, + "e": { + "customerID": "e", + "firstName": "Joel", + "lastName": "Griffiths" + }, + "f": { + "customerID": "f", + "firstName": "Michael", + "lastName": "Spinks" + }, + "g": { + "customerID": "g", + "firstName": "Victor", + "lastName": "Savkov" + } } ``` -Now to convert our customer dictionary to JSON, we need to convert our dictionary of customer objects to a dictionary of customers where each customer is a dictionary instead of a class.
-To do this we generate a new dictionary and populate it by converting each customer object to a dictionary : +Now that we have our customer data, we can read and update the data with our two simple functions: + ``` -customerDict = {} - -for id in customers: - customerDict[id] = customers[id].__dict__ - -``` -Then we can finally test our update and get functionality: - -``` -updateCustomers(customerDict) - customers = getCustomers() print(customers) ``` -Let's test it and see the `customers.json` file stored to our server and -retrieved. +Let's add a customer and write it back to file: + +``` +customers["h"] = Customer("h", "Marcel", "Dempers").__dict__ +updateCustomers(customers) +``` + +Now `customers.json` has the new entry if we rerun our code: ``` python src/app.py ``` -## External Libraries +## Flask +Checkout official Flask documentation [here](https://flask.palletsprojects.com/en/2.0.x/) - - -## JSON Library - -Python provides a library for dealing with JSON.
-Let's take our customer dictionary, and convert it to json structure and -finally write it to file using our update function. +A minimal flask application looks like this: ``` -import json +from flask import Flask + +app = Flask(__name__) + +@app.route("/") +def hello_world(): + return "

Hello, World!

" ``` -Now first things first, we need to convert our customers dictionary to a JSON structured string so we can store it to file.
+To get this to work, we need to look at Pythons Library system, or `pip` -The `json` library allows us to convert dictionaries to files using -the `dumps` function: +## External Packages + +If we look at the `from flask import Flask` statement, it assumes we have a `flask` dependency. Our previous import statements worked, because they are built-in libraries in python.
+Flask is an external package.
+External packages are managed by [pip](https://pypi.org/project/pip/) + +We may use `pip` to install `Flask`. If we need to connect to databases, we may install packages to do so. Like a `mysql` or a `redis` package.
+ +With `pip`, we define our dependencies in a `requirements.txt` file. +We can check for the latest version of `Flask` [here](https://pypi.org/project/Flask/) + +We can also use `pip install` commands to install dependencies.
+However, this means we need a new `pip install` command for every dependency which will grow as +our application and needs grow. Best is to use a `requirements.txt` file. + +Let's create a `requirements.txt` file: ``` -jsonData = json.dumps(customers) -print(jsonData) +Flask == 2.0.1 ``` -But if we run this we get an error: -`TypeError: Object of type Customer is not JSON serializable` - -That is because the library can only deal with full dictionaries. -Taking a closer look at our dictionary, our customer is a class. +We can install our dependencies using: ``` -# our key is a string -# our value is a class -"h" : Customer("h", "Marcel", "Dempers") +pip install -r src/requirements.txt ``` -`json.dumps()` only allows full dictionaries so we need our customer object to be in dictionary form like so: +Let's implement the minimal example we posted above.
+To run our application, note that the `app.py` runs as per usual.
+But to start `Flask`, we can follow the instructions as per the official document: ``` -"h": { "customerID": "h", "firstName" : "Marcel", "lastName" : "Dempers"} +export FLASK_APP=src/app +flask run -h 0.0.0 -p 5000 ``` -To fix our dictionary, we can iterate it and build a new one +Note that we cannot access `http://localhost:5000` because our app is running in docker.
+To access it, we need to restart our container and this time expose our port. ``` -# start with an empty one -customerDict = {} +exit +docker run -it -p 5000:5000 -v ${PWD}:/work python sh -#populate our new dictionary -for id in customers: - customerDict[id] = customers[id].__dict__ - -#print it -print(customerDict) +#get our dependencies and start our application +pip install -r requirements.txt +export FLASK_APP=src/app +flask run -h 0.0.0 -p 5000 ``` +Now we can access our app on `http://localhost:5000/` -Now we can convert our customer dictionary to json +## Routes + +Web servers allow us to define various URLs or routes.
+Currently we can see logs showing `GET` requests for route `/`
+Notice the HTTP status code `200` indicating success. ``` -customerJSON = json.dumps(customerDict) -print(customerJSON) + * Running on http://0.0.0:5000/ (Press CTRL+C to quit) +172.17.0.1 - - [22/Sep/2021 23:10:14] "GET / HTTP/1.1" 200 - +172.17.0.1 - - [22/Sep/2021 23:10:14] "GET /favicon.ico HTTP/1.1" 404 - +172.17.0.1 - - [22/Sep/2021 23:12:20] "GET / HTTP/1.1" 200 - +172.17.0.1 - - [22/Sep/2021 23:12:21] "GET / HTTP/1.1" 200 - +172.17.0.1 - - [22/Sep/2021 23:12:21] "GET / HTTP/1.1" 200 - ``` -Now we've converted our data into JSON, we can view this in a new tab window and change the language mode to JSON to visualise the data in our new format. - -Now essentially our application will use dictionaries when working with the data -but store and transport it as JSON.
- -Therefore our update function should take a dictionary and convert it to JSON -and store it. +Routes allow us to have many endpoints, so we could have endpoints such as: ``` -def updateCustomers(customer): - with open('customers.json', 'w', newline='') as customerFile: - customerJSON = json.dumps(customer) - customerFile.write(customerJSON) +/ --> which returns all customers +/get/ --> which returns one customer by CustomerID +/add --> which adds or updates a customer ``` -We also need to change our `getCustomers` function to read the JSON file -and convert that data correctly back to a dictionary for processing. +Defining these routes are pretty straight forward, see [docs](https://flask.palletsprojects.com/en/2.0.x/quickstart/#variable-rules) + +Let's define our routes: ``` -def getCustomers(): - if os.path.isfile("customers.json"): - with open('customers.json', newline='') as customerFile: - data = customerFile.read() - customers = json.loads(data) - return customers + +@app.route("/") +def get_customers(): + return "

Hello, get_customers!

" + +@app.route("/get/", methods=['GET']) +def get_customer(customerID): + return "

Hello, get!

" + +@app.route("/add", methods=['POST']) +def add_customer(customer): + return "

Hello, add!

" + +``` + +Now that we've built our URLs using routes, next we need to understand HTTP methods + +## HTTP Methods + +There are a number of HTTP methods for web services, popular ones being `POST` and `GET` + +`GET` method is for general retrieval of data
+`POST` method is for passing data to our service
+ +Let's setup each of our routes with dedicated HTTP Methods + +``` +@app.route("/", methods=['GET']) +def get_customers(): + return "

Hello, get_customers!

" + +@app.route("/get/", methods=['GET']) +def get_customer(customerID): + return "

Hello, get!

" + +@app.route("/add", methods=['POST']) +def add_customer(): + return "

Hello, add!

" + +``` + +Now we can see if we access `http://localhost:5000/add` +We get: `Method Not Allowed` , because it only accepts `POST` and browser by default, does `GET` + +Let's fill out our routes + +### Get Customers + +``` +@app.route("/", methods=['GET']) +def get_customers(): + customers = getCustomers() + return json.dumps(customers) +``` + +### Get Customer + +``` +@app.route("/get/", methods=['GET']) +def get_customer(customerID): + customer = getCustomer(customerID) + return customer +``` + +## HTTP Status Codes + +You can access your browser's developer tools and see the network tab
+Status codes are important for troubleshooting production traffic. + +`200` = Success.
+ +We can then access a customer via the `/get/` route. +Our Customer ID's are `a` to `h` and notice we get an `Internal Server Error` if +we pass an incorrect ID.
+ +`500` = Internal Server Error
+This means something went wrong in our code. See the logs indicating a `KeyError`, because we're trying to access a key in our customer dictionary that does not exist.
+ +Let's handle that error + +``` +#update our getCustomer function to check if the key exists + +def getCustomer(customerID): + customers = getCustomers() + if customerID in customers: + return customers[customerID] else: - return {} + return {} ``` -Finally let's test our functions : +Note when we rerun this, we get a blank customer back.
+Also note we're getting a `200` status code.
+This may not be super appropriate, as our customer we requested is technically not found. There is a status code for that
+ +`404` = Not Found
+ +Let's handle the status code. ``` -customers = getCustomers() -print(customers) +@app.route("/get/", methods=['GET']) +def get_customer(customerID): + customer = getCustomer(customerID) + if customer == {}: + return {}, 404 + else: + return customer ``` -It's also very easy to add a customer to our JSON data using our class and using pythons internal `__dict__` property to convert the object to a dictionary and add it to our customers dictionary +## Add Customer + +In order to add a customer, we need to be able to read request data.
+More about `Requests` [here](https://flask.palletsprojects.com/en/2.0.x/api/#flask.Request) + +Import requests library +``` +from flask import request +``` + +Read the request body as JSON: + +``` +@app.route("/add", methods=['POST']) +def add_customer(): + print(request.json) + return "success", 200 +``` +Let's test that! + +``` +curl -X POST http://localhost:5000/add -d '{ "data" : "hello" }' +``` +Notice our print message is `None`, this is because we need to specify content type in the request header. Let's send the content type as JSON. + +``` +curl -X POST \ +-H 'Content-Type: application/json' \ +http://localhost:5000/add -d '{ "data" : "hello" }' +``` + +Now we see that we read the data as `{'data': 'hello'}`
+This allows us to read customer data as JSON, validate it, and write it to storage.
+ +Let's `POST` a customer: + +``` +curl -X POST -v \ +-H 'Content-Type: application/json' \ +http://localhost:5000/add -d ' +{ + "customerID": "i", + "firstName": "Bob", + "lastName": "Smith" +}' +``` +Note if we break the `json` format, we get a `400` status code
+ +`400` = Bad Request
+ +Now we need to parse the request and validate it ``` -customers["i"] = Customer("i", "Bob", "Smith").__dict__ -updateCustomers(customers) ``` ## Docker @@ -249,10 +430,31 @@ Build and run our container. Notice the `customers.json` file gets created if it does not exist. ``` -cd python\introduction\part-3.json +cd python\introduction\part-4.http docker build . -t customer-app -docker run -v ${PWD}:/work -w /work customer-app +docker run -p 5000:5000 customer-app +``` + +Notice that we need to mount our `customers.json` file into the container.
+But also notice the containers working directory is `/app` and in the `app.py` +we load `customers.json` from the working directory which is the same directory as the app.
+When mounting files to a container, docker effectively formats that path, so to mount our json file, we would lose our `app.py` because of the format.
+ +Therefore it's always important to mount configs, secrets or data to seperate folders. + +Let's change that to `/data` + +``` +#set a global variable +dataPath = "/data/customers.json" +``` +Now we can change all references to `customers.json` to our variable. +And then mount that location: + +``` +docker build . -t customer-app +docker run -it -p 5000:5000 -v ${PWD}:/data customer-app ``` \ No newline at end of file diff --git a/python/introduction/part-4.http/dockerfile b/python/introduction/part-4.http/dockerfile index 7666e03..e4c7b93 100644 --- a/python/introduction/part-4.http/dockerfile +++ b/python/introduction/part-4.http/dockerfile @@ -3,6 +3,11 @@ FROM python:3.9.6-alpine3.13 as dev WORKDIR /work FROM dev as runtime -COPY ./src/ /app +WORKDIR /app +COPY requirements.txt /app/ +RUN pip install -r /app/requirements.txt -ENTRYPOINT [ "python", "/app/app.py" ] \ No newline at end of file +COPY ./src/ /app +ENV FLASK_APP=app.py + +CMD flask run -h 0.0.0 -p 5000 \ No newline at end of file diff --git a/python/introduction/part-4.http/requirements.txt b/python/introduction/part-4.http/requirements.txt new file mode 100644 index 0000000..8cee146 --- /dev/null +++ b/python/introduction/part-4.http/requirements.txt @@ -0,0 +1 @@ +Flask == 2.0.1 \ No newline at end of file diff --git a/python/introduction/part-4.http/src/app.py b/python/introduction/part-4.http/src/app.py index f0a986e..503fd6e 100644 --- a/python/introduction/part-4.http/src/app.py +++ b/python/introduction/part-4.http/src/app.py @@ -1,7 +1,10 @@ import os.path import csv import json +from flask import Flask +from flask import request +dataPath = "/data/customers.json" class Customer: def __init__(self, c="",f="",l=""): self.customerID = c @@ -11,8 +14,8 @@ class Customer: return self.firstName + " " + self.lastName def getCustomers(): - if os.path.isfile("customers.json"): - with open('customers.json', newline='') as customerFile: + if os.path.isfile(dataPath): + with open(dataPath, newline='') as customerFile: data = customerFile.read() customers = json.loads(data) return customers @@ -20,32 +23,44 @@ def getCustomers(): return {} def getCustomer(customerID): - customer = getCustomers() - return customer[customerID] + customers = getCustomers() + if customerID in customers: + return customers[customerID] + else: + return {} def updateCustomers(customers): - with open('customers.json', 'w', newline='') as customerFile: + with open(dataPath, 'w', newline='') as customerFile: customerJSON = json.dumps(customers) customerFile.write(customerJSON) +app = Flask(__name__) -customers = { - "a": Customer("a","James", "Baker"), - "b": Customer("b", "Jonathan", "D"), - "c": Customer("c", "Aleem", "Janmohamed"), - "d": Customer("d", "Ivo", "Galic"), - "e": Customer("e", "Joel", "Griffiths"), - "f": Customer("f", "Michael", "Spinks"), - "g": Customer("g", "Victor", "Savkov"), - "h" : Customer("h", "Marcel", "Dempers") -} +@app.route("/", methods=['GET']) +def get_customers(): + customers = getCustomers() + return json.dumps(customers) -customerDict = {} +@app.route("/get/", methods=['GET']) +def get_customer(customerID): + customer = getCustomer(customerID) + if customer == {}: + return {}, 404 + else: + return customer -for id in customers: - customerDict[id] = customers[id].__dict__ +@app.route("/add", methods=['POST']) +def add_customer(): + jsonData = request.json -updateCustomers(customerDict) + if "customerID" not in jsonData: + return "customerID required", 400 + if "firstName" not in jsonData: + return "firstName required", 400 + if "lastName" not in jsonData: + return "lastName required", 400 -customers = getCustomers() -print(customers) \ No newline at end of file + customers = getCustomers() + customers[jsonData["customerID"]] = Customer( jsonData["customerID"], jsonData["firstName"], jsonData["lastName"]).__dict__ + updateCustomers(customers) + return "success", 200 \ No newline at end of file