Getting Started

Lessons

Getting Started with Go Boilerplate

In this post, I'm not going to explain the in and outs of Go API development. I will provide you with a starting point and a basic walkthrough. This Go API is built off Go version 1.14. I'm assuming you have Mongo on your computer and running.

I have created a Go Boilerplate repository. I'd appreciate it if you would star it on Github.

It follows a handlers, models, repositories, services, and utils structure.

In short:

  • handlers handles the inbound request.
  • models contains all structs that represent data.
  • repositories has all methods for interacting with the database.
  • services contains the business logic of the app.
  • utils has any helper methods.

People have used many patterns for API code in general and Go code. I found this way keeps the code fairly organized and clear where to find something. It allows you to think about our API in layers. Something going wrong with a cars request and throwing a database error? It's probably in repositories/car_repository.go. Quick and simple. I should note this is NOT the traditional way of doing Go struct. Many project each "aspect" in its own package. I will come out with another tutorial for this approach.

I go into a fair amount of detail about each method. It may seem a little repetitive but many aspects of this CRUD app do not change. I decided to go for "boring and too much detail" over "quick and unclear". I will let you filter and skip. If you do have any questions, please reach out. I always love feedback and anyway to make this post better for others.

The starting point is main.go at the root. Starting at main(). We connect to the database. We initialize the repositories, services, and handlers. You can see the list of the API endpoints.

There are two sets of objects; users and cars. The user endpoints include:

  • Sign In
  • Sign Up
  • Log Out

The second set of endpoints is cars. I start you off with the basic endpoints:

  • List
  • Get
  • Create
  • Delete
  • Update

There are two middleware methods. The first one is CORSMiddleware. The second one is ValidateAuth. Validate auth checks for an authorization header. It validates the session is valid and exists. It then stores the session that has been retrieved in the Gin context. This avoids making a second call to the database in the actual endpoint code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
func ValidateAuth(userRepository repositories.UserRepository) gin.HandlerFunc {
	return func(c *gin.Context) {
		authToken := c.Request.Header.Get("Authorization")
		if authToken == "" {
			c.AbortWithStatus(403)
			return
		}

		authToken = strings.ReplaceAll(authToken, "Bearer ", "")

		found, session, err := userRepository.GetSessionById(authToken)
		if err != nil {
			fmt.Printf("err :: %+v\n", err)
			c.AbortWithStatus(403)
			return
		}
		if !found {
			fmt.Println("Not found")
			c.AbortWithStatus(403)
			return
		}
		c.Set("session", session)

		c.Next()
	}
}

Handlers

There are three files in handlers: cars_handler.go, user_handler.go and health_handler.go. Let's start with health_handler. It's very simple:

1
2
3
4
5
6
7
8
9
package handlers

import (
	"github.com/gin-gonic/gin"
)

func HealthCheck(c *gin.Context) {
	c.JSON(200, gin.H{"message": "Healthy"})
}

The second handler is user_handler.go. There is three methods to focus on: SignIn, SignUp and LogOut. Most methods are fairly simple, and all three are POST requests.

  1. It binds the request body to a struct object.
  2. It validates the struct based on the tags.
  3. Calls the service.
  4. Returns the JSON object.

For LogOut, it ignores the JSON request. We just need the current authentication token to figure out the session. We will update that session to be invalid.

The third handler is cars_handler.go. It contains all the methods for the carAPI from main.go. This is:

  • List
  • Get
  • Put/Update
  • Post/Create
  • Delete

For list, we have GetAll. It calls GetSession which pulls the session out of the Gin context. This was added here by the ValidateAuth method. This is a GET request and therefore any attributes come in using query parameters. There are a few optional onces:

  • Page
  • Limit
  • Make
  • Model
  • Year

Make, model and year are all related to filtering for a type of car. I show you where they are used.

Page and limit are used for paging. This involves only retrieve a subset of the list. It helps save on compute power, database resources, and keeps response times fast. A simple example is if you have 100 entries in your database. If you pull all 100 at once, it takes 5 seconds to response. If you pull 10, it takes 500 milliseconds. You add paging, and the user would pull the first 10 using page=1. If they find what they are looking for, you have wasted resources retrieving the other 490. You can read more about paging with a quick Google search.

In terms of a line like this:

1
page := c.DefaultQuery("page", "1")

The first attribute is the query key and the second is the default value. For page, limit, and year, we convert the strings to integers. This quickly allows us to understand if there are any issues with the query parameters provided.

All optional parameters are loaded into a struct. This makes it easier for managing. We can simply pass ListCarQuery throughout the app.

We run a validate function. I'm not completely sold to this command. It can be very powerful if you want to write your own. This function uses the struct tags to determine what is required, what form some values should value, etc. You can find a list of baked in validation on the projects README. An example:

1
2
3
type SearchQuery struct {
	Email   string `json:"email" validate:"required,email"`
}

The last parts of this GetAll method is calling the business logic that lives in services. It returns a list of cars or an error. If the error, is provided throw it. You can also change this to throw a generic 500. If cars is nil (like no documents found), we set it to be an empty list. This avoids { "cars": null, "error": "" } and makes it { "cars": [], "error": "" }.

For the next endpoint, GET - /cars/, it is tied to GetByID. If you look at the endpoint in main.go (or below), you can see the :id is provided in the URL.

1
carsAPI.GET("/:id", ValidateAuth(userRepository), carsHandler.GetByID)

The very first step in GetByID is to fetch this ID.

1
carsID := c.Param("id")

Next, we grab the session similar to GetAll. We call the business logic, and return the results or error.

The next method in the handler is Create. This is tied to the POST - /cars/. It will create a new instance of a car in the database. Similar to the other POST requests (Sign up, sign in), the steps have not changed. The first step it performs is capturing the request body and binding it to a struct. The second step is validating the input. This is a custom written validation method called Valid . You can find details in models/cars.go. This is the step I'm referencing:

1
2
3
4
if err := body.Valid(); err != nil {
	c.JSON(400, gin.H{"message": err.Error()})
	return
}

Following the validate method, we grab the session from the context. We have retrieved the session in the middleware and added it to the context. Lastly, we call the service for creating the new car. At this point, we have confirm the session, and validated the input as a struct type.

The Update method again is nothing different. It grabs the id using the parameter like GetByID. It then processes and validates the request body similar to Create.

The Delete method is similar is similar to GetByID. It does not require a request body. It just needs the ID and the session. From that, the API knows what document to delete and who owns the document.

Services

As mentioned before, the service layer is where the business logic lives. For example, a sign up endpoint would validate the user does not exist, create the new user, save them and sign them in.

We have two files: user_service.go and cars_service.go. Each have at least one method that corresponds with a handler. There may be some additional ones. I'm going to skip over the fairly straight forward methods (ie. Ones that return the repository method response).

Starting with user_service.go. We have SignUp, SignIn, and LogOut. In addition, we have signIn, GetEncryptedPassword, IsValidPassword.

I'll start by noting the difference between signIn and SignIn. Go capitalizes external facing methods and in this case Sign In. After a user signs up, they need to be signed in. I've moved the majority of sign in logic into this helper method called signIn. This allows both SignIn and SignUp to call it. You can rename it if it is too confusing.

The steps for sign up are fairly simple:

  1. Format email
  2. Verify password meets requirements
  3. Check for existing user (If exists, sign them in)
  4. Create new user
  5. Move onto sign in

This method is fairly straight forward. I do call a few external methods: GetEncryptedPassword and SaveUser. We use the User object from the models and fill in all required ones.

1
2
3
4
5
6
7
8
9
10
11
// Sign up user
newUser := models.User{
	Email:    emailTrimmed,
	Password: u.GetEncryptedPassword(body.Password),
	Name:     body.Name,
	Created:  time.Now(),
}
err = u.userRepository.SaveUser(newUser)
if err != nil {
	return "", err
}

The next method is SignIn. It simply deconstructs the struct and passes in the values as arguments. All of the heavily lifting is in the helper method.

1
2
3
4
5
func (u *UserService) SignIn(body models.SignInBody) (string, error) {
	emailLowerCase := strings.ToLower(body.Email)
	emailTrimmed := strings.Trim(emailLowerCase, " ")
	return u.signIn(emailTrimmed, body.Password)
}

signIn is where a lot of the logic lives. This is the helper method. It exists simply because when someone signs up, they also sign in. We do not want to duplicate the steps twice. Its fairly straight forward method:

  1. Get the encrypted password
  2. Grab the user
  3. Compared the stored encrypted password against the password provided
  4. Create a session and save it
  5. Return the session token.

A session has an expiry of one day but you can decrease that for better security.

1
expiryDate := now.AddDate(0, 0, 1)

The additional helper methods: GetEncryptedPassword and IsValidPassword. IsValidPassword is receiving an unencrypted password. You would add any password requirements checks here. Ex. Requires two numbers, minimum length and a special character. GetEncryptedPassword should use some sort of encryption library like bcrypt. You should not be storing passwords unencrypted.

The last method is LogOut. This service simply invalidates the current auth session. Using the auth token, we look up the session and call the mark as expired.

The second service is the cars_service.go. Most methods within this service have very little business logic and simply are a step between the repository layer. GetAll, GetRandom, GetByID, Update and Delete all take inputs from the handler and call their corresponding repository method. Create also does this but it creates a new Car object before passing it to carsRepository.Save.

Repositories

There are two repository files: user_repository.go and cars_repository.go. I will step through some of the steps in the user_repository.go. You will find that there are five possible actions that can be performed: query, query many, insert, update, and delete. Once you see the code for one, it doesn't need to be explained again. Ex. Save for an user and save for a car are the same. The Mongo Driver uses the bson Go tags.

SaveUser is an example of "insert" into a collection. Since in the model object we use bson tags, there is very little code required. In the save user case, we just care if there is an error.

1
2
3
4
5
6
7
func (u *UserRepository) SaveUser(user models.User) error {
	_, err := u.db.Collection("users").InsertOne(context.TODO(), user)
	if err != nil {
		return err
	}
	return nil
}

In SaveSession, we want to capture the documents new ID. We then convert it to a string because we know it will be returned in the response.

1
2
3
4
5
6
7
8
func (u *UserRepository) SaveSession(session models.Session) (string, error) {
	insertResult, err := u.db.Collection("sessions").InsertOne(context.TODO(), session)
	if err != nil {
		return "", err
	}

	return insertResult.InsertedID.(primitive.ObjectID).Hex(), nil
}

Next, we can look at an update call to the Mongo driver. In Update, we need two things. The changes and the filter.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
func (u *UserRepository) MarkSessionAsExpired(authToken string) error {
	docID, err := primitive.ObjectIDFromHex(authToken)
	if err != nil {
		return err
	}

	filter := bson.M{"_id": docID}
	update := bson.M{"$set": bson.M{"expiry": time.Now()}}
	_, err := u.db.Collection("sessions").UpdateOne(context.TODO(), filter, update)
	if err != nil {
		return err
	}
	return nil
}

In the code above, we filter by the _id.. For the update, anything inside the $set attribute will be updated. Mongo has lots of options in terms of actions (Ex. $push, $pop, etc.). You can find a list on the Mongo documentation.

We can switch to the cars_repository.go for more implementations of these methods. The Delete method is as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
func (c *CarsRepository) Delete(email string, carID string) error {
	docID, err := primitive.ObjectIDFromHex(carID)
	if err != nil {
		return err
	}

	filter := bson.M{"_id": docID, "email": email}
	_, err = c.db.Collection("cars").DeleteOne(context.TODO(), filter)
	if err != nil {
		return err
	}
	return nil
}

Once again fairly straight forward. We take a string and convert it to be an ObjectID. We then build a filter to find the object being deleted. We execute delete and we want to know if an error has been thrown.

We need to cover the two retrieval methods: get one item and get multiple. Both involve building a filter. Get multiple requires additional work like paging and iterating on a cursor, etc. First the get one:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
func (c *CarsRepository) Get(email string, carID string) (models.Car, error) {
	docID, err := primitive.ObjectIDFromHex(carID)
	if err != nil {
		return models.Car{}, err
	}

	filter := bson.M{"_id": docID, "email": email}

	var result models.Car
	err = c.db.Collection("cars").FindOne(context.TODO(), filter).Decode(&result)
	if err != nil {
		return models.Car{}, err
	}
	return result, nil
}

We decode the response into the models.Car struct item. Below is the get multiple:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
func (c *CarsRepository) List(email string, query models.ListCarQuery) ([]models.Car, error) {
	filters := query.Filter(email)
	var cars []models.Car

	options := options.Find()

	// Add paging
	options.SetLimit(int64(query.Limit))
	options.SetSkip(int64((query.Page * query.Limit) - query.Limit))

	// Add timestamp
	options.SetSort(bson.M{"created": -1})

	cursor, err := c.db.Collection("cars").Find(context.Background(), filters, options)
	if err != nil {
		return []models.Car{}, err
	}

	for cursor.Next(context.Background()) {
		car := models.Car{}
		err := cursor.Decode(&car)
		if err != nil {
			//handle err
		} else {
			cars = append(cars, car)
		}
	}
	return cars, nil
}

Once again, we build a filter. This code is housed next to the model struct. We initialize our empty array. We set a limit and skip for paging attributes. We sort the collections response so we can pull it up the same next time. We then iterate through using the cursor. It decodes it into a struct and adds it to the slice/array.

Those are the majority of repository functions you will see. All the basic Mongo actions are reused in different ways.

Models

The model represent the objects used through out the application. We store our structs here. They are organized by the categorization system as everything else (cars or users). Similar to the repository methods, I will not go through each one. I will explain anything new once.

You can open either file and see a struct used. For example in models/cars.go, you can see Cars:

1
2
3
4
5
6
7
8
9
type Car struct {
	ID      primitive.ObjectID `json:"id" bson:"_id,omitempty"`
	Make    string             `json:"make" bson:"make"`
	Model   string             `json:"model" bson:"model"`
	Year    int                `json:"year" bson:"year"`
	Status  string             `json:"status" bson:"status"`
	Email   string             `json:"email" bson:"email"`
	Created time.Time          `json:"created" bson:"created"`
}

When the JSON response is provided, it uses the json tag. When being stored in the database, it uses bson. We can create a "car" without an ID (hence the omitempty). When the car is added to the database Mongo will populate the document id.

I put a lot of my filters next to the struct. If you look in models/cars.go and look at ListCarQuery, you can see Filter below it. This method is fairly straight forward. We are trying to build a search query based on a number of arguments.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
func (q *ListCarQuery) Filter(email string) bson.M {
	andFilters := []bson.M{
		bson.M{
			"email": email,
		},
	}

	if q.Make != "" {
		orFilters := []bson.M{
			// Exact match
			bson.M{
				"make": q.Make,
			},
			// Similar match
			bson.M{
				"make": bson.M{
					"$regex": primitive.Regex{
						Pattern: "^" + q.Make + "*",
						Options: "i",
					},
				},
			},
		}

		andFilters = append(andFilters, bson.M{"$or": orFilters})
	}

	if q.Model != "" {
		orFilters := []bson.M{
			// Exact match
			bson.M{
				"model": q.Model,
			},
			// Similar match
			bson.M{
				"model": bson.M{
					"$regex": primitive.Regex{
						Pattern: "^" + q.Model + "*",
						Options: "i",
					},
				},
			},
		}

		andFilters = append(andFilters, bson.M{"$or": orFilters})
	}

	if q.Year != 0 {
		andFilters = append(andFilters, bson.M{"year": q.Year})
	}

	if len(andFilters) == 0 {
		// Handle empty and, since there must be one item.
		return bson.M{}
	}
	return bson.M{"$and": andFilters}
}

We start with a list of "ands". All must be true for the document to be valid. The first is that the email should exist. At a minimum, we can return just this query. However, if they provide additional attributes like model, make or year, we big to build a more complex query. We check for an exact match or a similar match and add it to the query. We do this for most attributes.

Utils

This project does not have any utility methods by default. I like to add a folder called utils for any third party integrations. For example, sending emails, uploading files to S3, etc. These utility methods would be on a similar level to the repository methods. They would be called by the services.

There is one utility method that does come in this project and that is db. The database folder houses all connections to the database. At this point, it's very basic. There are no passwords, dynamic strings for multiple environments, etc. It's a simple, connect to this host and port, and return the database connection instance.

Conclusion

That's it! Wow, I went in lots of depth in some parts and very little in others. The goal of this post is to help support my other Go posts that start with this project. There are going to be a few of them. If you have any questions, feel free to reach out. I'd love to add additional notes to the document.

Please go star the repository! Thanks for reading.

On This Page