How I'm testing MongoDB calls in Go (for now)
I started working on a little hobby project over this Easter vacation that makes use of MongoDB. But one thing I wasn’t sure about was how to test Mongo operations. Spanner provides the very helpful spannertest package. Redis has miniredis, allowing you to spin up a little Redis instance for each test. I was thinking I’d likely have to mock Mongo myself, but it turns out its Go driver comes with a helpful (but unstable!) mocking package called mtest. I learned about this in this post and suggest checking it out if you’re curious about this experimental mocking feature of mongo-go-driver.
I’m using mtest to mock responses for simple writes and reads. I haven’t used MongoDB itself in many years, so I’m just learning as I go along.
What I’m testing
I have a struct called Mongo:
type Mongo struct {
uri string
dbName string
client *mongo.Client
keyRing keyring.KeyRing
}
The Mongo struct implements the following Storage interface:
type Storage interface {
Connect(ctx context.Context) error
StoreActor(ctx context.Context, actor *actor.Actor) error
GetActor(ctx context.Context, id string) (*actor.Actor, error)
FindActorByPreferredName(ctx context.Context, preferredName string) (*actor.Actor, error)
}
The interface will expand somewhat as I work on the project, but for now that’s it.
I am currently testing the WIP StoreActor(), GetActor(), and FindActorByPreferredName() methods with the help of mtest.
Testing MongoDB writes
Here is the method being tested for storing an actor:
func (m *Mongo) StoreActor(ctx context.Context, actor *actor2.Actor) error {
// Generate keypair
if m.keyRing != nil {
pubKey, err := m.keyRing.GenerateKeyPair(actor.PreferredUsername)
if err != nil {
return fmt.Errorf("failed to generate key pair: %w", err)
}
// The public key is not actually stored in the DB, we set it
// for possible subsequent usage of the actor struct. May avoid this
// later and force caller to re-retrieve.
actor.PublicKey = actor2.NewPublicKey(actor.ID, pubKey)
} else {
logrus.Warn("No keyring specified. Ensure this is not running in production.")
}
collection := m.db.Collection(actorCollection)
res, err := collection.InsertOne(ctx, actor)
if err != nil {
return fmt.Errorf("%s: %w", err.Error(), ErrActorInsertFailed)
}
logrus.Infof("inserted actor: %v", res)
return nil
}
My StoreActor() test is defined as follows, with two basic test cases (which will be expanded later). I’ve decided to just inline the explanations as code comments:
// TestStoreActor tests storing an Actor instance in MongoDB
func TestStoreActor(t *testing.T) {
t.Parallel()
// Define test cases
testCases := []struct {
// Test case name
name string
// Get actor instance to store
getActor func(mt *mtest.T) *actor.Actor
// The response we expect Mongo to return
mongoRes bson.D
// Desired error
wantErr error
}{
{
name: "success",
getActor: func(mt *mtest.T) *actor.Actor {
// Just create a test actor and return it
a, err := actor.NewActor(actor.KindPerson, testDomain, "alyssa")
require.NoError(mt, err)
return a
},
// Empty success response
mongoRes: mtest.CreateSuccessResponse(),
},
{
name: "failure",
getActor: func(mt *mtest.T) *actor.Actor {
// Same actor as above
a, err := actor.NewActor(actor.KindPerson, testDomain, "alyssa")
require.NoError(mt, err)
return a
},
// Have Mongo return a write error
mongoRes: mtest.CreateWriteErrorsResponse(mtest.WriteError{
Index: 1,
Code: 123,
Message: "some error",
}),
wantErr: ErrActorInsertFailed,
},
}
opts := mtest.NewOptions().DatabaseName(testDbName).ClientType(mtest.Mock)
// Create instance of `mtest.T`, which will contain a
// test database.
mt := mtest.New(t, opts)
defer mt.Close()
for _, tc := range testCases {
// Shadow `tc` var to keep relevant test case in scope,
// since these tests run in parallel
tc := tc
// mt.Run() creates a new T and a new collection for
// the test case
mt.Run(tc.name, func(mt *mtest.T) {
mt.Parallel()
// Add our mock response (either the success
// or error as per test case definitions)
mt.AddMockResponses(tc.mongoRes)
// Create instance of our own Mongo struct, override the
// database with the test one we created via `mtest.New()`
mongo := NewMongo(testMongoURI, testDbName, nil)
mongo.db = mt.DB
actor := tc.getActor(mt)
// Call the method we are actually testing and ensure that
// we got the correct error
gotErr := mongo.StoreActor(context.Background(), actor)
require.ErrorIs(mt, gotErr, tc.wantErr)
})
}
}
The storage test was actually the simplest one to set up with the mocks, because I don’t expect any complex result structure. The main calls as they relate to our Mongo mock are instantiating our test db with mtest.New(), and setting up expected Mongo returns with mtest.Create*Response() and mtest.AddMockResponse()
Now, let’s look at getting an actor from the collection. This is where things get a bit more fiddly, since I have to mock cursor results.
Testing MongoDB reads
Here’s the method being tested here:
func (m *Mongo) FindActorByPreferredName(ctx context.Context, preferredName string) (*actor2.Actor, error) {
collection := m.db.Collection(actorCollection)
var actor actor2.Actor
if err := collection.FindOne(ctx, bson.D{{"preferredName", preferredName}}).Decode(&actor); err != nil {
if err == mongo.ErrNoDocuments {
return nil, ErrActorNotFound
}
return nil, fmt.Errorf("failed to get actor: %w", err)
}
// Get public key
publicKey, err := m.keyRing.GetPublicKey(actor.PreferredUsername)
if err != nil {
return nil, fmt.Errorf("failed to get public key: %w", err)
}
actor.PublicKey = actor2.NewPublicKey(actor.ID, publicKey)
return &actor, nil
}
And the test itself:
func TestFindActorByPreferredName(t *testing.T) {
t.Parallel()
// Define test cases
testCases := []struct {
name string
getActor func(mt *mtest.T, userName string) *actor.Actor
prepMongoMock func(mt *mtest.T, ring keyring.KeyRing, actor *actor.Actor)
userName string
wantErr error
}{
{
name: "success",
userName: "alyssa",
getActor: func(mt *mtest.T, userName string) *actor.Actor {
a, err := actor.NewActor(actor.KindPerson, testDomain, userName)
require.NoError(mt, err)
return a
},
prepMongoMock: func(mt *mtest.T, ring keyring.KeyRing, actor *actor.Actor) {
// Marshal actor to bson data
bsonData, err := bson.Marshal(actor)
require.NoError(mt, err)
// Generate test keypair
_, err = ring.GenerateKeyPair(actor.PreferredUsername)
require.NoError(mt, err)
var bsonD bson.D
// Unmarshal bson data to bson document
err = bson.Unmarshal(bsonData, &bsonD)
require.NoError(mt, err)
// Prep mock cursor responses
res := mtest.CreateCursorResponse(
1,
fmt.Sprintf("%s.%s", testDbName, testTblName),
mtest.FirstBatch,
bsonD)
// Mock cursor end
end := mtest.CreateCursorResponse(
0,
fmt.Sprintf("%s.%s", testDbName, testTblName),
mtest.NextBatch)
// Add cursor mocks above to responses
mt.AddMockResponses(res, end)
},
},
{
name: "not-found",
userName: "alyssa",
getActor: func(mt *mtest.T, userName string) *actor.Actor {
a, err := actor.NewActor(actor.KindPerson, testDomain, userName)
require.NoError(mt, err)
return a
},
prepMongoMock: func(mt *mtest.T, ring keyring.KeyRing, actor *actor.Actor) {
// Mock response with no data
res := mtest.CreateCursorResponse(
1,
fmt.Sprintf("%s.%s", testDbName, testTblName),
mtest.FirstBatch)
end := mtest.CreateCursorResponse(
0,
fmt.Sprintf("%s.%s", testDbName, testTblName),
mtest.NextBatch)
mt.AddMockResponses(res, end)
},
wantErr: ErrActorNotFound,
},
}
opts := mtest.NewOptions().DatabaseName(testDbName).ClientType(mtest.Mock)
mt := mtest.New(t, opts)
defer mt.Close()
for _, tc := range testCases {
tc := tc
mt.Run(tc.name, func(mt *mtest.T) {
mt.Parallel()
// getTestMongo creates a Mongo instance with
// a test database and a test keyring (to be
// covered below)
mongo := getTestMongo(mt)
defer mongo.keyRing.DeleteKeys()
// Get test case actor and prepare mongo mocks
actor := tc.getActor(mt, tc.userName)
tc.prepMongoMock(mt, mongo.keyRing, actor)
// Call the main function being tested
gotActor, gotErr := mongo.FindActorByPreferredName(context.Background(), actor.PreferredUsername)
require.ErrorIs(mt, gotErr, tc.wantErr)
if gotErr != nil {
return
}
// Verify returned fields
require.NotEmpty(mt, gotActor.PublicKey)
require.EqualValues(mt, actor.ID, gotActor.ID)
require.EqualValues(mt, actor.PreferredUsername, gotActor.PreferredUsername)
require.EqualValues(mt, actor.Kind, gotActor.Kind)
require.EqualValues(mt, actor.Name, gotActor.Name)
require.EqualValues(mt, actor.Icon, gotActor.Icon)
require.EqualValues(mt, actor.Summary, gotActor.Summary)
})
}
}
Above, I need to do some extra test mongo setup as well as mock cursor returns. Before storing each new actor, a keypair is generated. In case a key ring is not provided on actor storage, a warning is printed without a key being generated. But when retrieving an actor, I always expect to also retrieve and return its public key in the Actor struct. For that, Mongo needs to have a keyring (in this case a local keyring for testing purposes):
func getTestMongo(mt *mtest.T) *Mongo {
ring := getTestKeyring(mt)
mongo := NewMongo(testMongoURI, testDbName, ring)
mongo.db = mt.DB
return mongo
}
func getTestKeyring(mt *mtest.T) keyring.KeyRing {
keyLoc, err := ioutil.TempDir("", "k2test")
require.NoError(mt, err)
ring := keyring.NewLocalRing(keyLoc)
return ring
}
That’s it
So that’s where I am with testing mongo calls so far. I have not fully explored all of mtest yet. For example, I’m not sure of a way to specify expected requests (as opposed to just responses). Right now this is an experimental starting point.