Ocassional error when browsing online interactions

jei3

Not sure if this happened before the cloud changes (didn't browse online interactions that much before) but i'm noticing every once in a while i get a generic "an error has occurred" message in game when browsing interactions. When this happens the server returns a Code 500 (internal server error).

odes

Yeah I've noticed this as well. At seemingly random times, the server will just fail with a 500. This seems to be caused by either one of two things.

1. Database connection issue
2. Completely random error with no error message

In the first case, I'm already retrying to connect and execute a query up to 5 times (with a delay) if it for whatever reason fails at first. It's like Google Cloud SQL sometimes just refuses any connections so it becomes impossible to query the database. But if it still hasn't managed to get a connection by the 5th attempt, then the server will fail and cause the 500 error.

In the second case, I have no idea what's wrong. The error in the logs makes no sense and doesn't point out anything in the code.

https://gyazo.com/93b7408f91dc592364f9ea33544a16c9

For now I'm adding a retrying mechanism to the client as well, and will also make it possible to click "retry" in the error message so you don't have to start from page 0 if the server still fails to produce a result after several attempts at both server and client side.

jei3

yeah, the one I was seeing most looked like it might have been DB related

The server has either erred or is incapable of performing the requested operation.
    Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~divine-surface-93413/1.395962084796564181/main.py", line 685, in post
    rowcount = int(rowcount[0]['rows'])
KeyError: 'rows'

odes

Yeah, in this case it has failed to run a query so it doesn't have any results (even after 5 attempts). I think it's inherent to the cloud nature that services aren't 100% available. Looking at this answer:

http://stackoverflow.com/questions/25139344/google-app-engine-and-cloud-sql-lost-connection-to-mysql-server-at-reading-ini

It seems like it isn't uncommon behavior for GAE (OperationalError is an exception we catch and retry request if it happens.)

odes

I changed the code a little bit to do the exponential backoff thingie. So instead of just waiting 0.5 seconds in between each attempt, it waits 0.5 the first time, 1 the next, then 2, 4, and 8 for the other attempts. That might "fix" it.