How I'm testing MongoDB calls in Go (for now)

May 05, 2022 dev

I started working on a little hobby project over this Easter vacation that makes use of MongoDB. But one thing I wasn’t sure about was how to test Mongo operations. Spanner provides the very helpful spannertest package. Redis has miniredis, allowing you to spin up a little Redis instance for each test. I was thinking I’d likely have to mock Mongo myself, but it turns out its Go driver comes with a helpful (but unstable!) mocking package called mtest. I learned about this in this post and suggest checking it out if you’re curious about this experimental mocking feature of mongo-go-driver.

I’m using mtest to mock responses for simple writes and reads. I haven’t used MongoDB itself in many years, so I’m just learning as I go along.

What I’m testing

I have a struct called Mongo:

type Mongo struct {
	uri     string
	dbName  string
	client  *mongo.Client
	keyRing keyring.KeyRing
}

The Mongo struct implements the following Storage interface:

type Storage interface {
	Connect(ctx context.Context) error
	StoreActor(ctx context.Context, actor *actor.Actor) error
	GetActor(ctx context.Context, id string) (*actor.Actor, error)
	FindActorByPreferredName(ctx context.Context, preferredName string) (*actor.Actor, error)
}

The interface will expand somewhat as I work on the project, but for now that’s it.

I am currently testing the WIP StoreActor(), GetActor(), and FindActorByPreferredName() methods with the help of mtest.

Testing MongoDB writes

Here is the method being tested for storing an actor:

func (m *Mongo) StoreActor(ctx context.Context, actor *actor2.Actor) error {
	// Generate keypair
	if m.keyRing != nil {
		pubKey, err := m.keyRing.GenerateKeyPair(actor.PreferredUsername)
		if err != nil {
			return fmt.Errorf("failed to generate key pair: %w", err)
		}
		// The public key is not actually stored in the DB, we set it
		// for possible subsequent usage of the actor struct. May avoid this
		// later and force caller to re-retrieve.
		actor.PublicKey = actor2.NewPublicKey(actor.ID, pubKey)
	} else {
		logrus.Warn("No keyring specified. Ensure this is not running in production.")
	}

	collection := m.db.Collection(actorCollection)
	res, err := collection.InsertOne(ctx, actor)
	if err != nil {
		return fmt.Errorf("%s: %w", err.Error(), ErrActorInsertFailed)
	}
	logrus.Infof("inserted actor: %v", res)
	return nil
}

My StoreActor() test is defined as follows, with two basic test cases (which will be expanded later). I’ve decided to just inline the explanations as code comments:

// TestStoreActor tests storing an Actor instance in MongoDB
func TestStoreActor(t *testing.T) {
	t.Parallel()
	// Define test cases
	testCases := []struct {
		// Test case name
		name string
		// Get actor instance to store
		getActor func(mt *mtest.T) *actor.Actor
		// The response we expect Mongo to return
		mongoRes bson.D
		// Desired error
		wantErr error
	}{
		{
			name: "success",
			getActor: func(mt *mtest.T) *actor.Actor {
				// Just create a test actor and return it
				a, err := actor.NewActor(actor.KindPerson, testDomain, "alyssa")
				require.NoError(mt, err)
				return a
			},
			// Empty success response
			mongoRes: mtest.CreateSuccessResponse(),
		},
		{
			name: "failure",
			getActor: func(mt *mtest.T) *actor.Actor {
				// Same actor as above
				a, err := actor.NewActor(actor.KindPerson, testDomain, "alyssa")
				require.NoError(mt, err)
				return a
			},
			// Have Mongo return a write error
			mongoRes: mtest.CreateWriteErrorsResponse(mtest.WriteError{
				Index:   1,
				Code:    123,
				Message: "some error",
			}),
			wantErr: ErrActorInsertFailed,
		},
	}

	opts := mtest.NewOptions().DatabaseName(testDbName).ClientType(mtest.Mock)

	// Create instance of `mtest.T`, which will contain a
	// test database.
	mt := mtest.New(t, opts)
	defer mt.Close()

	for _, tc := range testCases {
		// Shadow `tc` var to keep relevant test case in scope,
		// since these tests run in parallel
		tc := tc

		// mt.Run() creates a new T and a new collection for
		// the test case
		mt.Run(tc.name, func(mt *mtest.T) {
			mt.Parallel()
			// Add our mock response (either the success
			// or error as per test case definitions)
			mt.AddMockResponses(tc.mongoRes)

			// Create instance of our own Mongo struct, override the
			// database with the test one we created via `mtest.New()`
			mongo := NewMongo(testMongoURI, testDbName, nil)
			mongo.db = mt.DB

			actor := tc.getActor(mt)

			// Call the method we are actually testing and ensure that
			// we got the correct error
			gotErr := mongo.StoreActor(context.Background(), actor)
			require.ErrorIs(mt, gotErr, tc.wantErr)
		})
	}
}

The storage test was actually the simplest one to set up with the mocks, because I don’t expect any complex result structure. The main calls as they relate to our Mongo mock are instantiating our test db with mtest.New(), and setting up expected Mongo returns with mtest.Create*Response() and mtest.AddMockResponse()

Now, let’s look at getting an actor from the collection. This is where things get a bit more fiddly, since I have to mock cursor results.

Testing MongoDB reads

Here’s the method being tested here:

func (m *Mongo) FindActorByPreferredName(ctx context.Context, preferredName string) (*actor2.Actor, error) {
	collection := m.db.Collection(actorCollection)
	var actor actor2.Actor
	if err := collection.FindOne(ctx, bson.D{{"preferredName", preferredName}}).Decode(&actor); err != nil {
		if err == mongo.ErrNoDocuments {
			return nil, ErrActorNotFound
		}
		return nil, fmt.Errorf("failed to get actor: %w", err)
	}
	// Get public key
	publicKey, err := m.keyRing.GetPublicKey(actor.PreferredUsername)
	if err != nil {
		return nil, fmt.Errorf("failed to get public key: %w", err)
	}
	actor.PublicKey = actor2.NewPublicKey(actor.ID, publicKey)
	return &actor, nil
}

And the test itself:

func TestFindActorByPreferredName(t *testing.T) {
	t.Parallel()
	// Define test cases
	testCases := []struct {
		name          string
		getActor      func(mt *mtest.T, userName string) *actor.Actor
		prepMongoMock func(mt *mtest.T, ring keyring.KeyRing, actor *actor.Actor)
		userName      string
		wantErr       error
	}{
		{
			name:     "success",
			userName: "alyssa",
			getActor: func(mt *mtest.T, userName string) *actor.Actor {
				a, err := actor.NewActor(actor.KindPerson, testDomain, userName)
				require.NoError(mt, err)
				return a
			},
			prepMongoMock: func(mt *mtest.T, ring keyring.KeyRing, actor *actor.Actor) {
				// Marshal actor to bson data
				bsonData, err := bson.Marshal(actor)
				require.NoError(mt, err)

				// Generate test keypair
				_, err = ring.GenerateKeyPair(actor.PreferredUsername)
				require.NoError(mt, err)

				var bsonD bson.D
				// Unmarshal bson data to bson document
				err = bson.Unmarshal(bsonData, &bsonD)
				require.NoError(mt, err)

				// Prep mock cursor responses
				res := mtest.CreateCursorResponse(
					1,
					fmt.Sprintf("%s.%s", testDbName, testTblName),
					mtest.FirstBatch,
					bsonD)

				// Mock cursor end
				end := mtest.CreateCursorResponse(
					0,
					fmt.Sprintf("%s.%s", testDbName, testTblName),
					mtest.NextBatch)

				// Add cursor mocks above to responses
				mt.AddMockResponses(res, end)
			},
		},
		{
			name:     "not-found",
			userName: "alyssa",
			getActor: func(mt *mtest.T, userName string) *actor.Actor {
				a, err := actor.NewActor(actor.KindPerson, testDomain, userName)
				require.NoError(mt, err)
				return a
			},
			prepMongoMock: func(mt *mtest.T, ring keyring.KeyRing, actor *actor.Actor) {
				// Mock response with no data
				res := mtest.CreateCursorResponse(
					1,
					fmt.Sprintf("%s.%s", testDbName, testTblName),
					mtest.FirstBatch)
				end := mtest.CreateCursorResponse(
					0,
					fmt.Sprintf("%s.%s", testDbName, testTblName),
					mtest.NextBatch)
				mt.AddMockResponses(res, end)
			},
			wantErr: ErrActorNotFound,
		},
	}
	opts := mtest.NewOptions().DatabaseName(testDbName).ClientType(mtest.Mock)
	mt := mtest.New(t, opts)
	defer mt.Close()
	for _, tc := range testCases {
		tc := tc

		mt.Run(tc.name, func(mt *mtest.T) {
			mt.Parallel()

			// getTestMongo creates a Mongo instance with
			// a test database and a test keyring (to be
			// covered below)
			mongo := getTestMongo(mt)
			defer mongo.keyRing.DeleteKeys()

			// Get test case actor and prepare mongo mocks
			actor := tc.getActor(mt, tc.userName)
			tc.prepMongoMock(mt, mongo.keyRing, actor)

			// Call the main function being tested
			gotActor, gotErr := mongo.FindActorByPreferredName(context.Background(), actor.PreferredUsername)
			require.ErrorIs(mt, gotErr, tc.wantErr)
			if gotErr != nil {
				return
			}

			// Verify returned fields
			require.NotEmpty(mt, gotActor.PublicKey)
			require.EqualValues(mt, actor.ID, gotActor.ID)
			require.EqualValues(mt, actor.PreferredUsername, gotActor.PreferredUsername)
			require.EqualValues(mt, actor.Kind, gotActor.Kind)
			require.EqualValues(mt, actor.Name, gotActor.Name)
			require.EqualValues(mt, actor.Icon, gotActor.Icon)
			require.EqualValues(mt, actor.Summary, gotActor.Summary)
		})
	}
}

Above, I need to do some extra test mongo setup as well as mock cursor returns. Before storing each new actor, a keypair is generated. In case a key ring is not provided on actor storage, a warning is printed without a key being generated. But when retrieving an actor, I always expect to also retrieve and return its public key in the Actor struct. For that, Mongo needs to have a keyring (in this case a local keyring for testing purposes):

func getTestMongo(mt *mtest.T) *Mongo {
	ring := getTestKeyring(mt)
	mongo := NewMongo(testMongoURI, testDbName, ring)
	mongo.db = mt.DB
	return mongo
}

func getTestKeyring(mt *mtest.T) keyring.KeyRing {
	keyLoc, err := ioutil.TempDir("", "k2test")
	require.NoError(mt, err)
	ring := keyring.NewLocalRing(keyLoc)
	return ring
}

That’s it

So that’s where I am with testing mongo calls so far. I have not fully explored all of mtest yet. For example, I’m not sure of a way to specify expected requests (as opposed to just responses). Right now this is an experimental starting point.