I began to write unit tests for my private project and ran into a weird problem that I have seen others comment on because of the inability to use fake HTTP requests for things like unit testing.

For particular App Engine operations, you pass the incoming HTTP request object to things like the Datastore. However, for unit tests, you have to create a fake HTTP request. Or so I thought.

This is the error that I get:

(Note: There are some solutions to this problem already before this update like this elegant solution by Mark Mendel. But now with the update it’s a lot simpler.)

However, version 1.9.11 of the Go App Engine SDK brought some very convenient additions. I will show you that a fairly recent update allows you fix this problem easily.

My Problem

I architected my code to be PaaS-independent because while I am using App Engine, in the future that may not be the case. For the majority (if not all App Engine transactions), you need a App Engine Context.

I architected the stack as following:

  1. The server receives a request and routes it to an appropriate function (e.g Login(*http.Request, http.ResponseWriter))
  2. Within the function, the incoming *http.Request is passed to some worker function that has a specific function and uses a specific technology (e.g. db.GetItem(“user_table”, username, *http.Request) – Getting a user from a database. The database technology can be different between PaaS and in the App Engine case we use Datastore.)
  3. This function resides in another file (sometimes even another package). I always have at least two files for these worker functions. Each file has its own implementation of the function and using build constraints I build only one of the files. Which one I build depends on the PaaS. (I have better abstract interface stuff going on but won’t go into detail for this post to keep it simple.)
  4. Within my App Engine specific file, I can take the *http.Request and create the context I need to use Datastore.

Now if I want to test from Step 2 onwards, I need to create my own incoming *http.Request. If I create my request and call the function within my unit test. I will ultimately get the error.

Road To The Fix

My reasoning was to check in the App Engine source code for why it’s panicking. So I just Googled the error and found the source code in api_dev.go

Ok so basically, I need to make sure that my request is in the ctxs map. Where do they normally do it?

As suspected, for every real request, they add it to the map.

Now, where else do they do it?

There’s this new function that adds what we need but it’s in the appengine_internal package.

If we search around the code in our “appengine” package we have…

Ah-ha. So if we can get an instance object and call NewRequest(), everything will be taken care of.

The Fix

My Go App Engine SDK is terribly out of date.

And that addition was just added to the version released 6 days ago as of this writing.

Time to upgrade!

Kudos to the Developers at Google who created the gcloud tool. It works so well.

Now lets look at the version.

Uh oh, the appengine version didn’t upgrade to version 1.9.11, which is the version we need. I checked the hosted json file, components-2.json, that the glcoud tool looks at and it looks that while they have released version 1.9.11 individually, they have not updated this file.

Update (Sept 30) – Until Sept 30, [I began to write unit tests for my private project and ran into a weird problem that I have seen others comment on because of the inability to use fake HTTP requests for things like unit testing.

For particular App Engine operations, you pass the incoming HTTP request object to things like the Datastore. However, for unit tests, you have to create a fake HTTP request. Or so I thought.

This is the error that I get:

(Note: There are some solutions to this problem already before this update like this elegant solution by Mark Mendel. But now with the update it’s a lot simpler.)

However, version 1.9.11 of the Go App Engine SDK brought some very convenient additions. I will show you that a fairly recent update allows you fix this problem easily.

My Problem

I architected my code to be PaaS-independent because while I am using App Engine, in the future that may not be the case. For the majority (if not all App Engine transactions), you need a App Engine Context.

I architected the stack as following:

  1. The server receives a request and routes it to an appropriate function (e.g Login(*http.Request, http.ResponseWriter))
  2. Within the function, the incoming *http.Request is passed to some worker function that has a specific function and uses a specific technology (e.g. db.GetItem(“user_table”, username, *http.Request) – Getting a user from a database. The database technology can be different between PaaS and in the App Engine case we use Datastore.)
  3. This function resides in another file (sometimes even another package). I always have at least two files for these worker functions. Each file has its own implementation of the function and using build constraints I build only one of the files. Which one I build depends on the PaaS. (I have better abstract interface stuff going on but won’t go into detail for this post to keep it simple.)
  4. Within my App Engine specific file, I can take the *http.Request and create the context I need to use Datastore.

Now if I want to test from Step 2 onwards, I need to create my own incoming *http.Request. If I create my request and call the function within my unit test. I will ultimately get the error.

Road To The Fix

My reasoning was to check in the App Engine source code for why it’s panicking. So I just Googled the error and found the source code in api_dev.go

Ok so basically, I need to make sure that my request is in the ctxs map. Where do they normally do it?

As suspected, for every real request, they add it to the map.

Now, where else do they do it?

There’s this new function that adds what we need but it’s in the appengine_internal package.

If we search around the code in our “appengine” package we have…

Ah-ha. So if we can get an instance object and call NewRequest(), everything will be taken care of.

The Fix

My Go App Engine SDK is terribly out of date.

And that addition was just added to the version released 6 days ago as of this writing.

Time to upgrade!

Kudos to the Developers at Google who created the gcloud tool. It works so well.

Now lets look at the version.

Uh oh, the appengine version didn’t upgrade to version 1.9.11, which is the version we need. I checked the hosted json file, components-2.json, that the glcoud tool looks at and it looks that while they have released version 1.9.11 individually, they have not updated this file.

Update (Sept 30) – Until Sept 30,](https://dl.google.com/dl/cloudsdk/release/components-2.json) stayed the same. I even filed an issue about it. The only way to update your SDK is by doing it manually. As of today, the json fille received an update and the Go SDK for App Engine is now version 1.9.12. (Which is perfect for our needs)!

Now we are using the right version!

You will be creating an instance manually and needs to be handled carefully on the resources side.

You have to:

  1. Manually call NewInstance()
  2. Manually call .Close() on that object. Preferably, by using the “defer” keyword.

 

Now, when you run your test via “goapp test”, you’ll see some new logging similar to when you run “goapp serve”.

 

That’s all folks! Until next time.