Lessons

How to Query Data using Regex from Mongo with Go

In this tutorial, I will show you how to query using regex in a Mongo database instance with Go code. I will be on local and provide you with all steps.

Setup

I'm assuming you have Mongo installed on your local machine. I also recommend installing Mongo Compass to make it easier to see your data.

I am assuming you have Go installed and have successful compiled at least one Go file. This would confirm your environment works. We will be using my Go boierplate code to save time on setting up our code structure. It follows a handlers, models, repositories, services, and utils structure.

In short:

• handlers handles the inbound request.
• models contains all structs that represent data.
• repositories has all methods for interacting with the database.
• services contains the business logic of the app.
• utils has any helper methods.

Download the Go boilerplate. Please give it a star on Github so I know people are still using and enjoying it. Move the files into your new project root. Open the files up in a text editor.

You need to do one thing for this boilerplate to work. Open services/user_service.go and within the IsValidPassword function. You will need to switch the variable to true. Do NOT use this code in production without fixing it. I leave encryption and password requirements up to you. This will store a plain text password till you add code to GetEncryptedPassword and IsValidPassword. For demo and dev purposes, this is fine.

You will need data in your database and checkout the insert tutorial.

Query Code

As you can imagine, we will be working with the repositories layer since it works with the database. If we open the repositories/cars_repository.go file, you can see the List method.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
func (c *CarsRepository) List(email string, query models.ListCarQuery) ([]models.Car, error) {
	filters := query.Filter(email)
	var cars []models.Car

	options := options.Find()

	// Add paging
	options.SetLimit(int64(query.Limit))
	options.SetSkip(int64((query.Page * query.Limit) - query.Limit))

	// Add timestamp
	options.SetSort(bson.M{"created": -1})

	cursor, err := c.db.Collection("cars").Find(context.Background(), filters, options)
	if err != nil {
		return []models.Car{}, err
	}

	for cursor.Next(context.Background()) {
		car := models.Car{}
		err := cursor.Decode(&car)
		if err != nil {
			//handle err
		} else {
			cars = append(cars, car)
		}
	}
	return cars, nil
}

In the code above, the filter comes from the query. If you open models/cars.go, you will see:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
type ListCarQuery struct {
	Page  int    `json:"page"`
	Limit int    `json:"limit"`
	Make  string `json:"make"`
	Model string `json:"model"`
	Year  int    `json:"year"`
}

func (q *ListCarQuery) Filter(email string) bson.M {
	andFilters := []bson.M{
		bson.M{
			"email": email,
		},
	}

	if q.Make != "" {
		orFilters := []bson.M{
			// Exact match
			bson.M{
				"make": q.Make,
			},
			// Similar match
			bson.M{
				"make": bson.M{
					"$regex": primitive.Regex{
						Pattern: "^" + q.Make + "*",
						Options: "i",
					},
				},
			},
		}

		andFilters = append(andFilters, bson.M{"$or": orFilters})
	}

	if q.Model != "" {
		orFilters := []bson.M{
			// Exact match
			bson.M{
				"model": q.Model,
			},
			// Similar match
			bson.M{
				"model": bson.M{
					"$regex": primitive.Regex{
						Pattern: "^" + q.Model + "*",
						Options: "i",
					},
				},
			},
		}

		andFilters = append(andFilters, bson.M{"$or": orFilters})
	}

	if q.Year != 0 {
		andFilters = append(andFilters, bson.M{"year": q.Year})
	}

	if len(andFilters) == 0 {
		// Handle empty and, since there must be one item.
		return bson.M{}
	}
	return bson.M{"$and": andFilters}
}

Let's just quickly go through what's happening. If there is no query parameters, the search should be:

1
WHERE email=<Email_provided_with_Auth>

Otherwise, we begin to build AND cases with OR cases within. What I mean by that is take make for example. It can be an exact match:

1
2
3
4
// Exact match
bson.M{
  "make": q.Make,
},

Or it can be similar (start with that string):

1
2
3
4
5
6
7
8
9
// Similar match
bson.M{
  "make": bson.M{
    "$regex": primitive.Regex{
      Pattern: "^" + q.Make + "*",
      Options: "i",
    },
  },
},

The pattern is your regex query. In this case, I'm saying if the word starts with it. I did this because car makes are usually only one word. If someone is trying to type gm or general motors and they get as far as g. Then it should not return all makes that contain a g. Ex. volkswagen. The user is probably not searching for volkswagen by typing in a middle letter.

In terms of the output of this query, it's:

1
2
3
4
5
6
WHERE
  email=<Email from Auth>
  AND
    make=g
    OR
    make=g*

The same search query is true for model. For year, it's fairly binary. Either they have the correct year or the wrong year. If they provide 20, then it can be from 2000 to 2099 (ie. all years).

Demo

In your terminal:

1
go run main.go

The first step is to run sign up or sign in. I execute my sign up request with Postman. Copy down the token in the response.

Or you can run this curl command:

1
2
3
4
5
6
curl --location --request POST 'http://localhost:8080/user/signup/' \
--header 'Content-Type: application/json' \
--data-raw '{
 "email": "me@keithweaver.ca",
 "password": "demodemo1"
}'

Write down the token. Connect to your database and confirm that you have multiple values.

Next, you will submit a GET request using a few incomplete make or model names.

Or by curl:

1
2
3
curl --location --request GET 'http://localhost:8080/cars/?model=f-typ' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <YOUR_TOKEN>'

And you can see it returns the car based on make. You can do the same with model:

Thanks for reading!