marko/docker-development-youtube-series
1
0
Fork 0
mirror of https://github.com/marcel-dempers/docker-development-youtube-series.git synced 2025-06-06 17:01:30 +00:00
Code Issues Packages Projects Releases Wiki Activity

part 1,2,3

Browse Source
This commit is contained in:
marcel-dempers 2021-01-04 21:17:54 +11:00
parent 776e42da90
commit 75e20b239f
16 changed files with 1008 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
golang/introduction/dockerfile
View File

@ -1,8 +1,8 @@
FROM golang:1.15 as dev
FROM golang:1.15-alpine as dev
WORKDIR /work
FROM golang:1.15 as build
FROM golang:1.15-alpine as build
WORKDIR /app
COPY ./app/* /app/

3
golang/introduction/part-2.json/dockerfile Normal file
View File

@ -0,0 +1,3 @@
FROM golang:1.15-alpine as dev
WORKDIR /work

324
golang/introduction/part-2.json/readme.md Normal file
View File

@ -0,0 +1,324 @@
# Introduction to Go: JSON
In programming languages, you will very often deal with data structures internally. <br/>
Sometimes, you need to pass data outside of your application or read data from another application, or even a file. <br/>
API, for example often expose data in `json`, `xml`, `grpc` and all sorts of formats.
Before we can really take a look at building APIs and even storing data in databases,
we need some fundamental knowledge about how to convert these data structures, into structures
that our application can understand. <br/>
In part 1, we dealt with [Variables]("https://tour.golang.org/basics/8") and more importantly, <br/> we dealt with `struct` data type, which made us build a `customer` object.
If we wanted to pass the `customer` to a database, or an external system, it's often required that we convert this to `json` format.
## Dev Environment
The same as Part 1, we start with a [dockerfile](./dockerfile) where we declare our version of `go`.
```
cd golang\introduction\part-2.json
docker build --target dev . -t go
docker run -it -v ${PWD}:/work go sh
go version
```
## Create our App
Create a new directory that holds defines our `repository` and holds our `module`
```
mkdir videos
```
* Define a module path
```
# change directory to your application source code
cd videos
# create a go module file
go mod init videos
```
## Create our base code
We start out all our applications with a `main.go` defining our `package`, declaring our `import` dependencies <br/>
and our entrypoint `main()` function
```
package main
import "fmt"
func main() {
fmt.Println("Hello, world.")
}
```
## Create our Videos app
Firstly, we create a seperate code file `videos.go` that deals with our YouTube videos <br/>
Similar to Part 1, we define the file, we define what a video looks like using a `struct` <br/>
and a function for returning a slice of videos.
```
package main
import ()
type video struct {
Id string
Title string
Description string
Imageurl string
Url string
}
func getVideos()(videos []video){
//Get our videos here,
//and return them
return videos
}
```
## Populating our Video Struct
Similar to our customers app, we created a `struct` and populated it with data:
Let's add the following to our `getVideos()` function:
```
video1 := video{
Id : "eyvLwK5C2dw",
Title : "Kubernetes on Azure",
Imageurl : "https://i.ytimg.com/vi/eyvLwK5C2dw/mqdefault.jpg?sqp=CISC_PoF&rs=AOn4CLDo7kizrJozB0pxBhxL9JbyiW_EPw",
Url : "https://youtu.be/eyvLwK5C2dw",
Description : "",
}
video2 := video{
Id : "QThadS3Soig",
Title : "Kubernetes on Amazon",
Imageurl : "https://i.ytimg.com/vi/QThadS3Soig/sddefault.jpg",
Url : "https://youtu.be/QThadS3Soig",
Description : "",
}
return []video{ video1, video2}
```
We need to also invoke our function that will return the videos
Don't worry about where the videos will come from. <br/>
We always start with the building blocks and move on from there <br/>
In our `main()` function: <br/>
```
func main() {
videos := getVideos()
fmt.Println(videos)
}
```
## Files
Now, Ideally, we do not want to "hard code" our videos like this. <br/>
Currently, our videos are embedded into the code and we can only return 2 videos. <br/>
If we want to introduce a new video, we have to rebuild the application. <br/>
In the real world, videos are our data. <br/>
Data lives outside of the application. Like in a database.
Technically, we can use a database, but that is too big of a step for now. <br/>
So let's start with a file instead.
### Introducing ioutil
It's important to learn how to navigate and read go documentation.
Let's go to https://golang.org/pkg/io/ioutil/
We can import this library since its part of the Go standard library set.
```
import (
"io/ioutil"
)
```
Let's move our videos into a `json` file called `videos.json` :
```
[
{
"id" : "QThadS3Soig",
"title" : "Kubernetes on Amazon",
"imageurl" : "https://i.ytimg.com/vi/QThadS3Soig/sddefault.jpg",
"url" : "https://youtu.be/QThadS3Soig",
"description" : ""
},
{
"id" : "eyvLwK5C2dw",
"title" : "Kubernetes on Azure",
"imageurl" : "https://i.ytimg.com/vi/eyvLwK5C2dw/mqdefault.jpg?sqp=CISC_PoF&rs=AOn4CLDo7kizrJozB0pxBhxL9JbyiW_EPw",
"url" : "https://youtu.be/eyvLwK5C2dw",
"description" : ""
}
]
```
## Reading a file
Let's take another look at https://golang.org/pkg/io/ioutil/ <br/>
Notice the `ReadFile` function
We can read a file from disk:
```
fileBytes, err := ioutil.ReadFile("./videos.json")
if err != nil {
panic(err)
}
fileContent := string(fileBytes)
fmt.Println(fileContent)
```
## Panic
Also notice the function `panic`. <br/>
"A panic typically means something went unexpectedly wrong. Mostly we use it to fail fast on errors that shouldn’t occur during normal operation, or that we aren’t prepared to handle gracefully."
- https://gobyexample.com/panic
For now, we will panic on every potential error. <br/>
In a future video, we'll cover Error handling in more depth
## JSON
Working with JSON is pretty straightforward in go. <br/>
`struct` objects are pretty well synergized with `json`
Let's take a look at the `json` package that is part of go.
https://golang.org/pkg/encoding/json/
```
"encoding/json"
```
This package allows us to marshal and unmarshal JSON. <br/>
This allows conversion between a `struct` or a go type, and `json`
## Unmarshal
From JSON(bytes) ==> Struct\Go type
```
err = json.Unmarshal(fileBytes, &videos)
if err != nil {
panic(err)
}
return videos
```
## Using our Data
Now in the real world, you would use your data to do something. <br/>
Let's say we'd like to update some common terms and conditions to the video description <br/>
In our `main` function, let's loop the videos & update the description.
Feel free to checkout Part 1 of the series on `loops`!
```
for _, video := range videos {
}
```
Now we dont want to override the description, we want to inject the terms and conditions on a new line
```
video.Description = video.Description + "\n Remember to Like & Subscribe!"
```
Note that when we run this, it does not print our new description field </br>
This is because the loop `range` is giving us a copy of the video object. </br>
This means we are updating a copy, but printing out the original. </br>
We need to use the loop indexer and update the original object in the slice. </br>
```
for i, _ := range videos {
videos[i].Description = videos[i].Description + "\n Remember to Like & Subscribe!"
}
```
Run our app again and now it updates the original video!
## Marshal
https://golang.org/pkg/encoding/json/#Marshal
From Struct\Go type ==> JSON(bytes)
Let's create a new function for saving our video back to file
```
func saveVideos(videos []video)(){
videoBytes, err := json.Marshal(videos)
if err != nil {
panic(err)
}
}
```
## Writing to File
https://golang.org/pkg/io/ioutil/
```
err = ioutil.WriteFile("./videos-updated.json", videoBytes, 0644)
if err != nil {
panic(err)
}
```
Then in our `main` function we can save our videos after the update:
```
saveVideos(videos)
```
## Mapping fields
In our example our `json` and our struct fields match exactly.
Sometimes this is not possible to maintain.
Sometimes we need to tell go, to map certain `json` fields to certain fields in our `struct`
We can do it like so:
```
type video struct {
Id string `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
Imageurl string `json:"imageurl"`
Url string `json:"url"`
}
```

3
golang/introduction/part-2.json/videos/go.mod Normal file
View File

@ -0,0 +1,3 @@
module videos
go 1.15

17
golang/introduction/part-2.json/videos/main.go Normal file
View File

@ -0,0 +1,17 @@
package main
import "fmt"
func main() {
videos := getVideos()
for i, _ := range videos {
videos[i].Description = videos[i].Description + "\n Remember to Like & Subscribe!"
}
fmt.Println(videos)
saveVideos(videos)
}

16
golang/introduction/part-2.json/videos/videos-updated.json Normal file
View File

@ -0,0 +1,16 @@
[
{
"id": "QThadS3Soig",
"title": "Kubernetes on Amazon",
"description": "\n Remember to Like \u0026 Subscribe!",
"imageurl": "https://i.ytimg.com/vi/QThadS3Soig/sddefault.jpg",
"url": "https://youtu.be/QThadS3Soig"
},
{
"id": "eyvLwK5C2dw",
"title": "Kubernetes on Azure",
"description": "\n Remember to Like \u0026 Subscribe!",
"imageurl": "https://i.ytimg.com/vi/eyvLwK5C2dw/mqdefault.jpg?sqp=CISC_PoF\u0026rs=AOn4CLDo7kizrJozB0pxBhxL9JbyiW_EPw",
"url": "https://youtu.be/eyvLwK5C2dw"
}
]

44
golang/introduction/part-2.json/videos/videos.go Normal file
View File

@ -0,0 +1,44 @@
package main
import (
"io/ioutil"
"encoding/json"
)
type video struct {
Id string `json:"id"`
Title string `json:"title"`
Description string `json:"description"`
Imageurl string `json:"imageurl"`
Url string `json:"url"`
}
func getVideos()(videos []video){
fileBytes, err := ioutil.ReadFile("./videos.json")
if err != nil {
panic(err)
}
err = json.Unmarshal(fileBytes, &videos)
if err != nil {
panic(err)
}
return videos
}
func saveVideos(videos []video){
videoBytes, err := json.Marshal(videos)
if err != nil {
panic(err)
}
err = ioutil.WriteFile("./videos-updated.json", videoBytes, 0644)
if err != nil {
panic(err)
}
}

16
golang/introduction/part-2.json/videos/videos.json Normal file
View File

@ -0,0 +1,16 @@
[
{
"id" : "QThadS3Soig",
"title" : "Kubernetes on Amazon",
"imageurl" : "https://i.ytimg.com/vi/QThadS3Soig/sddefault.jpg",
"url" : "https://youtu.be/QThadS3Soig",
"description" : ""
},
{
"id" : "eyvLwK5C2dw",
"title" : "Kubernetes on Azure",
"imageurl" : "https://i.ytimg.com/vi/eyvLwK5C2dw/mqdefault.jpg?sqp=CISC_PoF&rs=AOn4CLDo7kizrJozB0pxBhxL9JbyiW_EPw",
"url" : "https://youtu.be/eyvLwK5C2dw",
"description" : ""
}
]

14
golang/introduction/part-3.http/dockerfile Normal file
View File

@ -0,0 +1,14 @@
FROM golang:1.15-alpine as dev
WORKDIR /work
FROM golang:1.15-alpine as build
WORKDIR /videos
COPY ./videos/* /videos/
RUN go build -o videos
FROM alpine as runtime
COPY --from=build /videos/videos /
COPY ./videos/videos.json /
CMD ./videos

3
golang/introduction/part-3.http/go.mod Normal file
View File

@ -0,0 +1,3 @@
module videos
go 1.15

433
golang/introduction/part-3.http/readme.md Normal file
View File

@ -0,0 +1,433 @@
# Introduction to Go: HTTP
HTTP is a fundamental part of Microservices and Web distributed systems <br/>
Go has a built in HTTP web server package. The package can be found [here](https://golang.org/pkg/net/http/) <br/>
We simply have to import the `http` package:
```
import (
"net/http"
)
```
[In part 1](../readme.md), we covered the fundamentals of writing basic Go <br/>
[In part 2](../part-2.json/readme.md), we've learn how to use basic data structures like `json` so we can send\receive data. <br/>
We'll be combining both these techniques so we can serve our `videos` data over a web endpoint.
As always, let's start with our `dockerfile` , `main.go` and `videos.go` we created in Part 2
## Dev Environment
The same as Part 1+2, we start with a [dockerfile](./dockerfile) where we declare our version of `go`.
```
cd golang\introduction\part-3.http
docker build --target dev . -t go
docker run -it -v ${PWD}:/work go sh
go version
```
## Create our App
Create a new directory that holds defines our `repository` and holds our `module`
```
mkdir videos
```
* Define a module path
```
# change directory to your application source code
cd videos
# create a go module file
go mod init videos
```
## Create our base code
We start out all our applications with a `main.go` defining our `package`, declaring our `import` dependencies <br/>
and our entrypoint `main()` function
```
package main
import "fmt"
func main() {
fmt.Println("Hello, world.")
}
```
## Create our Videos app
Firstly, we create a seperate code file `videos.go` that deals with our YouTube videos <br/>
The `videos.go` file defines what a video `struct` looks like, a `getVideos()` function to retrieve <br/>
videos list as a slice and a `saveVideos()` function to save videos to a file locally. <br/>
Let's copy the following content from Part 2 and create `videos.go` :
We want `videos.go` to be part of package main:
```
package main
```
We import 2 packages, 1 for reading and writing files, and another for dealing with `json`
```
import (
"io/ioutil"
"encoding/json"
)
```
Then we define what a video `struct` looks like:
```
type video struct {
Id string
Title string
Description string
Imageurl string
Url string
}
```
We have a function for retrieving `video` objects as a list of type `slice` :
```
func getVideos()(videos []video){
fileBytes, err := ioutil.ReadFile("./videos.json")
if err != nil {
panic(err)
}
err = json.Unmarshal(fileBytes, &videos)
if err != nil {
panic(err)
}
return videos
}
```
We also need to copy our `videos.json` file which contains our video data. <br/>
And finally, we have a function that accepts a list of type `slice` and stores the videos to a local file
```
func saveVideos(videos []video)(){
videoBytes, err := json.Marshal(videos)
if err != nil {
panic(err)
}
err = ioutil.WriteFile("./videos-updated.json", videoBytes, 0644)
if err != nil {
panic(err)
}
}
```
## HTTP Package
https://golang.org/pkg/net/http/
The HTTP package allows us to implement an HTTP client and a server.
A client is a component that makes HTTP calls.
A server is a component that receives or serves HTTP.
The HTTP package is capable of sending HTTP requests as well as defining a server
for receiving HTTP requests.
We can use this to run an HTTP server to serve files, or serve data, like an API.
Let's define a server in `main.go` :
```
# just one line :)
http.ListenAndServe(":8080", nil)
# ListenAndServe starts an HTTP server with a given address and handler.
# The handler is usually nil, which means to use DefaultServeMux.
# Handle and HandleFunc add handlers to DefaultServeMux
```
Now before we run this, since we're running in Docker, we want to exit the container <br/>
and rerun it, but this time open port `8080`
```
docker run -it -p 8080:8080 -v ${PWD}:/work go sh
cd videos
go run main.go
# you will notice the application pausing
```
We should see our server with a 404 on http://localhost:8080/
## Handle HTTP requests
In order to handle requests, we can tell the HTTP service that we want it to run a function </br>
for the request coming in.
We can see the `http` package has a `HandleFunc` function: https://golang.org/pkg/net/http/
To see this in action, lets create a `Hello()` function:
```
func Hello(){
}
```
And tell our `http` service to run it:
```
http.HandleFunc("/", Hello)
```
We cannot run this yet. As per `http` documentation, our `Hello` function needs to take in some inputs.
`func HandleFunc(pattern string, handler func(ResponseWriter, *Request))`
Therefore we need to add inputs to our function:
```
func Hello(w http.ResponseWriter, r *http.Request){
}
```
This allows us to get the request, its `body`, `headers` and a write where we can send a response.
Run this in the browser and you will notice the 404 goes away, but we now get an empty response.
## HTTP Response
Let's write a reponse to the incoming request. </br>
The response write has a `Write()` function that takes a bunch of bytes. <br/>
We can convert string to bytes by casting a `string` to a `[]byte` <br/> like:
`[]byte("Hello!")`. Let's convert it and write "Hello" to the response:
```
w.Write([]byte("Hello!"))
```
IF we run this code, we can see "Hello!" in the response body
## HTTP Headers
Headers play an important role in HTTP communication. </br>
Lets access all the headers of the incoming request!
If we look at the Header definition [here](https://golang.org/pkg/net/http), we can see how to access it.
Let's use the `for` loop we learnt in [part 1](../readme.md)
```
for i, value := range r.Header {
}
```
We learn't from our loop, we have in indexer and a value.
For `i`, we can rename it to header since it represents the header key in the dictionary.
And the `value` is the value of type `[]string`, containing the value of the header:
```
for header, value := range r.Header {
fmt.Printf("Key: %v \t Value: %v \n", header, value)
}
```
We can use `fmt` to print out the values and look at the headers.
We can also set headers on our response. <br/>
If we take a look at the `http` docs, we can see header is also a dictionary or strings.
```
w.Header().Add("TestHeader", "TestValue")
```
You can now see the headers in the response value if you use `curl` or your browser development tools
## HTTP Methods | GET
Web servers can serve data in a number of ways and support multiple type of HTTP methods.
`GET` is used to request data from a specified resource.
So far, our HTTP route for our Hello function is using the `GET` method.
Let's make our `GET` method more useful by serving our video data </br>
Let's rename our `Hello()` function to `HandleGetVideos()`. </br>
Our `/` route will return all videos:
```
videos := getVideos()
```
In [part 2](../part-2.json/readme.md) we covered `JSON`.
We need to convert our video `slice` of `struct`'s to `JSON` in order to return it to the client.
For this we learnt about the Marshall function:
Import the `JSON` package:
```
"encoding/json"
```
Convert our videos to `JSON` :
```
videoBytes, err := json.Marshal(videos)
if err != nil {
panic(err)
}
w.Write(videoBytes)
```
If we run this code and hit our `/` endpoint, we can now see `JSON` data being returned. <br/>
This is a core part of building an API in Go. <br/>
## HTTP Methods | POST
A `POST` method is used to send data to a server to create/update a resource. <br/>
Since we built a `saveVideos` function, lets use that so a client can update videos! <br/>
We need to define a new route, we can all it `/update` :
```
http.HandleFunc("/update", HandleUpdateVideos)
```
And we need to define an `HandleUpdateVideos()` function:
```
func HandleUpdateVideos(w http.ResponseWriter, r *http.Request){
}
```
Let's validate the request method to ensure its `POST`
We need to also ensure we send a status code to inform the user of method not allowed.
https://golang.org/pkg/net/http/#ResponseWriter
```
if r.Method == "POST" {
//update our videos here!
} else {
w.WriteHeader(405)
fmt.Fprintf(w, "Method not Supported!")
}
```
Now we need to accept `JSON` from the `POST` request body
https://golang.org/pkg/net/http/#Request
In the docs above, we can see the request Body is of type `Body io.ReadCloser`
To read that, we can use the `ioutil` package
https://golang.org/pkg/io/ioutil/
```
import "io/ioutil"
```
Then we can read the body into a `slice` of `bytes`:
```
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
```
Now that we have the body in a `[]byte`, we need to use our knowledge from [part 2](../part-2.json/readme.md) where we <br/>
convert `[]byte` to a `slice` of `video` items.
```
var videos []video
err = json.Unmarshal(body, &videos)
if err != nil {
panic(err)
}
```
Creating our video objects allows us to do some validation if we wanted to. <br/>
We can ensure the request body adheres to our API contract for this videos API. <br/>
So instead of calling `panic`, lets return a `400` Bad request status code if we cannot <br/>
Unmarshal the `JSON` data. This might help with some basic validation.
```
w.WriteHeader(400)
fmt.Fprintf(w, "Bad request")
```
And Finally, let's update our videos file! :
```
saveVideos(videos)
```
# Build our Docker container
Let's uncomment all the build lines in the `dockerfile`
Full `dockerfile` :
```
FROM golang:1.15-alpine as dev
WORKDIR /work
FROM golang:1.15-alpine as build
WORKDIR /videos
COPY ./videos/* /videos/
RUN go build -o videos
FROM alpine as runtime
COPY --from=build /videos/videos /
COPY ./videos/videos.json /
CMD ./videos
```
Build :
```
cd golang\introduction\part-3.http
docker build . -t videos
```
Run :
```
docker run -it -p 8080:8080 videos
```
## Things to know
* SSL for secure web connection
* Authentication
* Good API validation
* Support a backwards compatible contract (Inputs remain consistent)

54
golang/introduction/part-3.http/videos/main.go Normal file
View File

@ -0,0 +1,54 @@
package main
import (
"net/http"
"encoding/json"
"io/ioutil"
"fmt"
)
func main() {
http.HandleFunc("/", HandleGetVideos)
http.HandleFunc("/update", HandleUpdateVideos)
http.ListenAndServe(":8080", nil)
}
func HandleGetVideos(w http.ResponseWriter, r *http.Request){
videos := getVideos()
videoBytes, err := json.Marshal(videos)
if err != nil {
panic(err)
}
w.Write(videoBytes)
}
func HandleUpdateVideos(w http.ResponseWriter, r *http.Request){
if r.Method == "POST" {
body, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
var videos []video
err = json.Unmarshal(body, &videos)
if err != nil {
w.WriteHeader(400)
fmt.Fprintf(w, "Bad request")
}
saveVideos(videos)
} else {
w.WriteHeader(405)
fmt.Fprintf(w, "Method not Supported!")
}
}

16
golang/introduction/part-3.http/videos/videos-updated.json Normal file
View File

@ -0,0 +1,16 @@
[
{
"Id": "QThadS3Soig",
"Title": "Kubernetes on Amazon",
"Description": "TEST",
"Imageurl": "https://i.ytimg.com/vi/QThadS3Soig/sddefault.jpg",
"Url": "https://youtu.be/QThadS3Soig"
},
{
"Id": "eyvLwK5C2dw",
"Title": "Kubernetes on Azure",
"Description": "TEST",
"Imageurl": "https://i.ytimg.com/vi/eyvLwK5C2dw/mqdefault.jpg?sqp=CISC_PoF\u0026rs=AOn4CLDo7kizrJozB0pxBhxL9JbyiW_EPw",
"Url": "https://youtu.be/eyvLwK5C2dw"
}
]

46
golang/introduction/part-3.http/videos/videos.go Normal file
View File

@ -0,0 +1,46 @@
package main
import (
"io/ioutil"
"encoding/json"
)
type video struct {
Id string
Title string
Description string
Imageurl string
Url string
}
func getVideos()(videos []video){
fileBytes, err := ioutil.ReadFile("./videos.json")
if err != nil {
panic(err)
}
err = json.Unmarshal(fileBytes, &videos)
if err != nil {
panic(err)
}
return videos
}
func saveVideos(videos []video)(){
videoBytes, err := json.Marshal(videos)
if err != nil {
panic(err)
}
err = ioutil.WriteFile("./videos-updated.json", videoBytes, 0644)
if err != nil {
panic(err)
}
}

16
golang/introduction/part-3.http/videos/videos.json Normal file
View File

@ -0,0 +1,16 @@
[
{
"id" : "QThadS3Soig",
"title" : "Kubernetes on Amazon",
"imageurl" : "https://i.ytimg.com/vi/QThadS3Soig/sddefault.jpg",
"url" : "https://youtu.be/QThadS3Soig",
"description" : ""
},
{
"id" : "eyvLwK5C2dw",
"title" : "Kubernetes on Azure",
"imageurl" : "https://i.ytimg.com/vi/eyvLwK5C2dw/mqdefault.jpg?sqp=CISC_PoF&rs=AOn4CLDo7kizrJozB0pxBhxL9JbyiW_EPw",
"url" : "https://youtu.be/eyvLwK5C2dw",
"description" : ""
}
]

2
golang/introduction/readme.md
View File

@ -213,7 +213,7 @@ fmt.Println(customer)
## Arrays
At the moment, we can only return 1 customer at a time on our function. <br/>
Realisticly we need the ability to return more data, not just a single customer. <br/>
Realistically we need the ability to return more data, not just a single customer. <br/>
Arrays allow us to make a collection of variables of the same type. <br/>
We can now return a list of customers! <br/>