Rafael Sanches

reducing appengine server costs

I will try to describe below how we decreased our server costs to half.

I already assume that you cache everything as possible on a double layer cache: memory + memcache.

Move your image serving outside of google “cdn”

Our website is heavy on images and we observed that Google CDN was billing us $120 per Terabyte.

We have moved our image serving to MaxCDN and paid $800 for 25 Terabytes. It means that we started paying $32 per Terabyte. It’s almost 4x less.

You may say that 4x isn’t much, but tell me if you prefer to pay $12.000,00 or $3.200,00?

Now, to do this we had to implement google’s “=s300-c” type of image conversion, which was good for us because we also were able to add a height crop type of transformation: “=s300=h200”, for example: http://c4.picmix.net/4955638627041280=s612=h300

Transfer only necessary data on your JSON Api’s

Same as the previous topic. Output bandwidth is really expensive, so try to transfer only the necessary.

Never update a transactional object in a loop

This seems to be quite simple, but it may happen if you don’t pay attention. Just observe your write/read ratio and see if there’s any inconsistency.

We had an old task that was working fine and at some point we changed our transactional configuration and it started to penalize us for weeks until we realized our bill had increased because of database writes.

On my case we had a task that would do something like this:

MyObject obj = pm.getObjectById(MyObject.class, 1);
while (i < 1000) {
  obj.addToField(randomValue);
  JDOHelper.makeDirty(obj, "field");
}
pm.close();

Changing the code to something like this made the cron 100x faster and decreased our write costs:

MyObject obj = pm.getObjectById(MyObject.class, 1);
List field = new ArrayList<>(obj.getField());
while (i < 1000) {
  field.add(randomValue);
}
obj.setField(field);
JDOHelper.makeDirty(obj, "field");
pm.close();

Notice that the first loop may not penalize you depending on your database transactional settings.

Now, I do understand that this may be clear for everyone. Although, it may happen when you’re distracted and appengine will penalize you both on performance and on write costs.

Use modules and split your frontend code and endpoints into multiple modules

This is necessary as appengine is sensitive to boot time and, in java, boot time is strictly related to classpath scanning.

Use frontend modules instead of backends. They’re eligible for frontend discount hours.

It’s also generally easier to individuate problems when your modules are separated. Specially because appengine give you different statistics dashboard for each module.

Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

%d bloggers like this: