Over the last couple of days I restructured SnailLife Go into a server and client. I’m still in the “rough draft” stage, but the top level project structure now looks like this:

gosnaillife
├── client
├── cmd
├── common
├── LICENSE.md
├── README.md
├── server
└── setup

Intent

  • Split application into server and client CLI apps.
  • Create REST API for client-server communication
  • Have a “common” package for structures which will be reused by both server and client
  • Create some rudimentary deployment scripts for both server and client

main.go

I started by creating snaillifesrv/main.go alongside snaillifecli/main.go. The cmd directory now looks like this:

cmd
├── snaillifecli
│   └── main.go
└── snaillifesrv
    └── main.go

snaillifecli/main.go

The client main.go runs some simple configuration with viper (right now there is just a server.json config file with the server url to connect to depending on which environment you are running). After running the configuration it waits for user input. Once input is received, it tries to find and run a cobra command by that name.

package main

import "fmt"
import (
	"os"
	"bufio"
	"gitlab.com/drakonka/gosnaillife/client/lib/interfaces/cli"
	"gitlab.com/drakonka/gosnaillife/client/lib/interfaces/cli/commands"
	"runtime"
	"strings"
	"path/filepath"
	"io/ioutil"
	"github.com/spf13/viper"
	"gitlab.com/drakonka/gosnaillife/common/util"
)

func main() {
	fmt.Println("Welcome to SnailLife! The world is your oyster.")
	configureClient()
	if err := commands.RootCmd.Execute(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
	waitForInput()
}

func configureClient() {
	projectRoot := getProjectRootPath()
	confPath := projectRoot + "/config"
	envname, err := ioutil.ReadFile(confPath + "/env.conf")
	if err != nil {
		util.HandleErr(err, "")
	}
	envFile := string(envname)
	configPath := confPath + "/" + envFile

	viper.AddConfigPath(configPath)

	// Config client
	viper.SetConfigName("server")
	err = viper.ReadInConfig()
	if err != nil {
		util.HandleErr(err, "")
	}
}

func waitForInput() {
	buf := bufio.NewReader(os.Stdin)
	fmt.Print("> ")
	input, err := buf.ReadBytes('\n')
	if err != nil {
		fmt.Println(err)
	} else {
		cmd, err := cli.TryGetCmd(string(input))
		if err != nil {
			fmt.Println(err)
		} else {
			err := cmd.Execute()
			if err != nil {
				fmt.Println("ERROR: " + err.Error())
			}
		}
	}
	waitForInput()
}

func getProjectRootPath() string {
	_, b, _, _ := runtime.Caller(0)
	folders := strings.Split(b, "/")
	folders = folders[:len(folders)-2]
	path := strings.Join(folders, "/")
	basepath := filepath.Dir(path) + "/client"
	return basepath
}

snaillifesrv/main.go

When launching snaillifesrv, a subcommand is expected immediately. Right now the only supported subcommand is serve, which will start the server.

package main

import "fmt"
import (
	"gitlab.com/drakonka/gosnaillife/server/lib/infrastructure/env"
	"os"
	"runtime"
	"path/filepath"
	"strings"
	"gitlab.com/drakonka/gosnaillife/server/lib/infrastructure"
	"gitlab.com/drakonka/gosnaillife/common"
	"gitlab.com/drakonka/gosnaillife/server/lib/interfaces/cli/commands"
)

var App env.Application

func main() {
	setProjectRootPath()
	confPath := env.ProjectRoot + "/config"
	App = infrastructure.Init(confPath, common.CLI)
	if err := commands.RootCmd.Execute(); err != nil {
		fmt.Println(err)
		os.Exit(1)
	}
}

func setProjectRootPath() {
	_, b, _, _ := runtime.Caller(0)
	folders := strings.Split(b, "/")
	folders = folders[:len(folders)-2]
	path := strings.Join(folders, "/")
	basepath := filepath.Dir(path) + "/server"
	env.ProjectRoot = basepath
}

Client

So far an extremely barebones implementation, it looks like this:

client
├── config
│   ├── config.go
│   ├── dev
│   │   └── server.json
│   └── env.conf
└── lib
    └── interfaces
        └── cli
            ├── cli.go
            ├── cmd.go
            └── commands
                ├── register.go
                ├── root.go
                └── test.go

Right now only the register command is implemented. The process looks something like this:

SnailLife CLI client registration

Server

The server is where the bulk of the existing packages ended up going:

server
├── config
│   ├── config.go
│   ├── dev
│   │   ├── auth.json
│   │   └── database.json
│   └── env.conf
└── lib
    ├── domain
    │   ├── item
    │   └── snail
    │       ├── snail.go
    │       └── snailrepo.go
    ├── infrastructure
    │   ├── auth
    │   │   ├── authenticator.go
    │   │   ├── auth.go
    │   │   ├── cli
    │   │   │   ├── auth0
    │   │   │   │   ├── auth0.go
    │   │   │   │   └── tests
    │   │   │   │       ├── auth0_test.go
    │   │   │   │       └── config_test.go
    │   │   │   ├── cli.go
    │   │   │   └── cli.so
    │   │   ├── provider.go
    │   │   └── web
    │   ├── databases
    │   │   ├── database.go
    │   │   ├── mysql
    │   │   │   ├── delete.go
    │   │   │   ├── insert.go
    │   │   │   ├── mysql.go
    │   │   │   ├── retrieve.go
    │   │   │   ├── tests
    │   │   │   │   └── mysql_test.go
    │   │   │   └── update.go
    │   │   ├── repo
    │   │   │   ├── repo.go
    │   │   │   ├── tests
    │   │   │   │   ├── repo_test.go
    │   │   │   │   ├── testmodel_test.go
    │   │   │   │   └── testrepo_test.go
    │   │   │   └── util.go
    │   │   └── tests
    │   │       └── testutil.go
    │   ├── env
    │   │   └── env.go
    │   ├── init.go
    │   └── init_test.go
    └── interfaces
        ├── cli
        │   └── commands
        │       ├── root.go
        │       └── serve.go
        └── restapi
            ├── err.go
            ├── handlers
            │   └── user.go
            ├── handlers.go
            ├── logger.go
            ├── restapi.go
            ├── router.go
            └── routes.go

I followed a lot of the advice from this useful post about creating REST APIs in Go. When the user runs the register command on the client, here is what happens on the server. I have added comments to the copy below to help explain:

package handlers


import (
	"encoding/json"
	"fmt"
	"errors"
	"io/ioutil"
	"io"
	"net/http"
	"gitlab.com/drakonka/gosnaillife/common/restapi"
	"gitlab.com/drakonka/gosnaillife/common/util"
	"strings"
	"gitlab.com/drakonka/gosnaillife/server/lib/infrastructure/auth"
	"gitlab.com/drakonka/gosnaillife/server/lib/infrastructure/env"
	http2 "gitlab.com/drakonka/gosnaillife/common/util/http"
)

func CreateUser(w http.ResponseWriter, r *http.Request) {
	fmt.Println("Creating user")
	var user restapi.UserReq
	body, err := ioutil.ReadAll(io.LimitReader(r.Body, 1048576))
    // The infamous Go error handling - I need a better way.
	if err != nil {
		util.HandleErr(err, "CreateUserErr")
		return
	}
	if err := r.Body.Close(); err != nil {
		util.HandleErr(err, "CreateUserErr")
		return
	}

    // Unmarshal the data we get from the client into UserReq
	if err := json.Unmarshal(body, &user); err != nil {
        // If we were unable to unmarshal, send an error response back to the client
		util.HandleErr(err, "CreateUserErr")
		w.Header().Set("Content-Type", "application/json; charset=UTF-8")
		w.WriteHeader(422) // unprocessable entity
		if err := json.NewEncoder(w).Encode(err); err != nil {
			util.HandleErr(err, "CreateUser")
			return
		}
		return
	}

	fmt.Println("Running registration")
	resBody, err := registerUser(user)
	if err != nil {
		util.HandleErr(err, "CreateUserErr")
	}

    // Start creating a userRes to send back to the client.
	userRes := buildUserResponse(resBody)
	status := http.StatusOK
	if err != nil {
		status = http.StatusInternalServerError
	}
	w.Header().Set("Content-Type", "application/json; charset=UTF-8")
	w.WriteHeader(status)
	if err := json.NewEncoder(w).Encode(userRes); err != nil {
		util.HandleErr(err, "CreateUserErr")
		return
	}
}

func registerUser(user restapi.UserReq) (resBody []byte, err error) {
    // Find an Auth0 provider (that is all we'll support for now)
	var auth0 auth.Provider
	auth0 = env.App.Authenticator.FindProvider("Auth0")
	if auth0 != nil {
		resBody, err = auth0.Register(user.Username, user.Password)
	} else {
		err = errors.New("Auth0 provider not found")
	}
	return resBody, err
}

func buildUserResponse(resBody []byte) (*restapi.UserRes) {
	res := restapi.UserRes{}
    // Find any keys we may find relevant from the Auth0 response body
	m, _ := util.FindInJson(resBody, []string {"_id", "statusCode", "name", "description", "error"})
	httpErr := buildHttpErr(m)
	if id, ok := m["_id"]; ok {
		res.Id = fmt.Sprintf("%v", id)
	}
	res.HttpErr = httpErr
	return &res
}

func buildHttpErr(m map[string]interface{}) (httpErr http2.HttpErr) {
    // The Auth0 response body *sometimes* contains errors in statusCode/name/description format and *sometimes* just contains a single "error" json key
	if sc, ok := m["statusCode"]; ok {
		codeStr := fmt.Sprintf("%v", sc)
		if strings.HasPrefix(codeStr,"4") || strings.HasPrefix(codeStr, "5") {
			scf := sc.(float64)
			httpErr.StatusCode = int(scf)
			httpErr.Name = fmt.Sprintf("%v", m["name"])
			httpErr.Desc = fmt.Sprintf("%v", m["description"])
		}
	} else if error, ok := m["error"]; ok {
		httpErr.StatusCode = 500
		httpErr.Name = "Error"
		httpErr.Desc = fmt.Sprintf("%v", error)
	}
	return httpErr
}

In the end the server sends a UserRes back to the client

package restapi

import (
	"gitlab.com/drakonka/gosnaillife/common/util/http"
)


type UserRes struct {
	HttpErr http.HttpErr `json:"httpErr"`
	Id string `json:"id"`
	Username string `json:"username"`
}

type UserReq struct {
	Username string `json:"username"`
	Password string `json:"password"`
	Connection string `json:"connection"`
}

Deployment

I made a couple of quick scripts to deploy client and server. Note that go-bindata lets you compile your config files into the binary, making for easier distribution (and maybe slightly improved security for the secret keys stored in the server config since you don’t have loose configs with credentials sitting around)

Client

#!/bin/sh

echo "Building and installing SnailLife"
go-bindata -o ../../client/config/config.go ../../client/config/...
cd ../../cmd/snaillifecli;
go build
GOBIN=$GOPATH/bin go install

Server

#!/bin/sh

echo "Building and installing SnailLife server"
go-bindata -o ../../server/config/config.go ../../server/config/...
cd ../../server/lib/infrastructure/auth/cli

echo "Building cli.so auth plugin"
go build -buildmode=plugin -o cli.so

echo "Building SnailLifeSrv"
cd ../../../../../cmd/snaillifesrv;
go build
GOBIN=$GOPATH/bin go install

Anyway, as you can see there is a long way to go. Up next I am going to write some tests for the REST API and the cobra commands (which I should really have been doing already).