Hands-on Android: lessons learned from our first game project

About one year ago we wanted to gain some experience in android app development and the whole associated workflow, consisting of:

  • developing an Android app
  • publishing and maintaining the app in the Play Store
  • performing marketing and earning some money

So we decided to create a trial-balloon-project to get started. Despite the fact that our company normally creates serious business software, we gladly received the ok to do some more “funny stuff” like a game for this special case. We are working for a really cool company, do we?
In this short series of blog posts I want to share with you some of the key-insights we’ve gained (and problems we’ve encountered) since the first version of the app was released one year ago. Today I want to start from a technical perspective which must be a little “game specific”. But the later blog posts will focus on the Google Play Store as well as marketing and selling activities we’ve tried, which should make the whole story more interesting for a wider audience as just game developers. So keep on reading! Continue reading

Strange multiselect in Flex lists

With old halo lists mx:List you can allow the user to select multiple list items. Set allowMultipleSelection = true and you can add single items to the selected group via Ctrl + Click. Via Shift + Click you can select a set of items. Click on one item to select the first and then Shift + Click on another one to select the whole range of items between the first and last one (each included).

Nothing unexpected so far.

But if you do another Shift + Click to change the selected range, the Flex list reacts strangely. For example, say you’ve clicked on item 3 and then Shift + clicked on item 5, you select items 3 to 5. But what happens when you Shift + click on item 7 afterwards? Windows Explorer and MacOS X Finder will select items 3 to 7. If you then Shift + click on 1, they select 1 to 3. So, the general rule for those standard item lists is that you set an anchor with the first click and each following Shift + click selects all items between this anchor and the last clicked item.

But not so in the Flex list. mx:List resets the anchor with every Shift + click to the last clicked item. The following table highlights the described differences between Explorer/Finder and the Flex list:

Action Explorer Finder mx:List
click 3 3 3 3
shift+click 5 3-5 3-5 3-5
shift+click 7 3-7 3-7 5-7
shift+click 1 1-3 1-3 1-7

I think this is very confusing for the user, because there is no obvious reason for the flex lists to behave in a different way than the corresponding native lists of the operating system. So I wrote the following patch to fix this:

import mx.collections.CursorBookmark;
import mx.controls.List;
import mx.controls.listClasses.IListItemRenderer;

public class OSLikeList extends List {

    private var firstShiftClick:Boolean = true;
    private var oldAnchorBookmark:CursorBookmark;
    private var oldAnchorIndex:int;

    public function OSLikeList() {
        super();
    }

    override protected function selectItem(item:IListItemRenderer, shiftKey:Boolean, ctrlKey:Boolean, transition:Boolean = true):Boolean {
        if (shiftKey && allowMultipleSelection) {
            if (firstShiftClick) {
                oldAnchorBookmark = anchorBookmark;
                oldAnchorIndex = anchorIndex;
                firstShiftClick = false;
            } else {
                anchorIndex = oldAnchorIndex;
                anchorBookmark = oldAnchorBookmark;
            }
        } else {
            firstShiftClick = true;
        }
        return super.selectItem(item,  shiftKey, ctrlKey, transition);
    }

}

How to install Sun Java JDK via command line and wget

Our business software REWOO Scope runs as a Java application and uses the Sun Java JDK under CentOS 7 (minimal package). To setup a linux system with a running Sun Java, you need a browser and working filetransfer – but why not use wget to download the missing file?

Simple question, difficult answer: Sun dissalowed direct downloads of java from their servers. You have to agree with the license conditions by browser.
It seems that a single cookie is all that is needed to bypass this. If you want to download jdk7u67 for 64-bit Linux using wget, you can use:

wget --no-cookies --no-check-certificate\
--header "Cookie: gpw_e24=http%3A%2F%2Fwww.oracle.com%2F; oraclelicense=accept-securebackup-cookie"\
"http://download.oracle.com/otn-pub/java/jdk/7u67-b01/jdk-7u67-linux-x64.rpm"

But remember: each version of Java is located in a different folder. For other versions than 7u67 you have to use the respective URL.

Flex application not redrawing screen

From time to time we observe issues with the Pepper player, the built-in Flash player of the Chrome browser. Since January we fight against the problem that our Flex application is not properly redrawing the screen after user interaction.

Fortunately, we stumbled across this issue in the chromium bug database. According to the comments there we have to force the refresh of the application. Of course, we should do this for the Pepper player only, but this is not a great deal. The Adobe online help states how to detect a Pepper player.

Solution

First we check if the user has a Pepper player. In this case we create a timer which calls every 500 ms invalidateDisplayList() on the main Application object.

private static var _refreshTimer:Timer;

public static function checkPepperPlayer(app:Application):void {
    if (Capabilities.manufacturer === "Google Pepper" && !_refreshTimer) {
        _refreshTimer = new Timer(500);
        _refreshTimer.addEventListener(TimerEvent.TIMER, function(event:TimerEvent):void {
            app.invalidateDisplayList()
        });
        _refreshTimer.start();
    }
}

Creating MS Word reports with Java

If your application stores a lots of user content, it seems to be an obvious idea to offer some template mechanism, which can be used by your customers to create reports. So, the user can get a printable, styled and easy to process version of the interesting information out of your system, designed by himself and filled with data with just one click. Of course, creating and styling those templates should be easy enough for your customer to do it by himself… so, whats about using the well-known MS Word application for it? The user creates a document with his beloved office suite from Redmond, containing formatted text, images and charts and finally adds also some placeholders. Afterwards he passes this document to your application and, whenever the user wants to create a report, you magically fill the placeholders with the actual data from your system. Reporting can be so easy … if you can find some way to manipulate the word document the correct way with the tools your platform offers!

Continue reading

Postgres down after Time Machine restore

A few days ago my MacBook crashed. I had to erase the harddisk and restore the data from the latest Time Machine backup. After a few hours reading a book my MacBook worked as though nothing had happened – except the postgres server. On OS X the log files can be found in /Library/PostgreSQL/8.4/data/pg_log

I looked in the newest log file with

sudo less /Library/PostgreSQL/8.4/data/pg_log/postgresql-2014-01-14_152823.log

and found a PANIC log message

2014-01-14 15:28:23 CET LOG:  record with zero length at 19/3DA0E2C0
2014-01-14 15:28:23 CET LOG:  invalid primary checkpoint record
2014-01-14 15:28:23 CET LOG:  record with zero length at 19/3DA0E27C
2014-01-14 15:28:23 CET LOG:  invalid secondary checkpoint record
2014-01-14 15:28:23 CET PANIC:  could not locate a valid checkpoint record

If you want to analyse the transaction log switch user to postgres and use

sudo su - postgres
/Library/PostgreSQL/8.4/bin/pg_resetxlog /Library/PostgreSQL/8.4/data

In my case the transaction log was corrupt and the data didn’t matter. So I used -f to force reset

/Library/PostgreSQL/8.4/bin/pg_resetxlog -f /Library/PostgreSQL/8.4/data

See also this discussion at stackoverflow

Free OnSite SEO Offline Tool

Recently we revamped our company’s website rewoo.de. We dropped our Rails based CMS and in favour to use the static site generator Jekyll. Jekyll gives us a great flexibility and extensibility, while the site load gets blazing fast. Not only the asset management with support for less and coffee script is a big win, we were able to implement all missing pieces quickly. E.g. we created new liquid template tags for email obfuscating, google analytics, or youtube video.

rewoo.de-screenshot

So the page was done. Now we need publicity, reputation and SEO – everyone needs SEO. While the OffSite SEO with all the back links, social media and other authorities is a very difficult topic, we could do some homework on OnPage SEO. OnPage SEO evaluates your single page and gives you some recommendations how to improve your page. Some rules are:

  • Have only one h1 heading
  • Write keywords and description in your HTML header
  • All images should have an alt attribute
  • title and h1 header should be different
  • All internal links to anchors should be valid
  • Text to HTML markup ratio should be high

There are many online SEO services out there like seitwert.de where you can check one page for free or pay a lot to get a full report. This was not an option. Back to DIY work. Validating OnPage SEO for one page is fine. However, having more than 30 pages this work is very time consuming. Further, manual work is fragile, error-prone and regress-prone. Why not doing this by a computer program?

Unfortunately, we did not find any appropriate offline tool which can crawl your site and gives some improvement suggestions for all our pages (if you know any, please leave a comment). A small afternoon hack later a little tool called OnSite SEO was born.

OnSite SEO is a tool to crawl, inspect and score your site offline. It inspects each page and extracts some key properties such meta information, headers, resources and text. These properties are scored by a list of different rating functions to give a final score of your page. Also you can validating links, check images and do other arbitrary calculations, if you know javascript. The tool is based on nodejs, phantomjs, jQuery, and angularjs.

onpage-seo-ratings

Since it is Christmas time we published this tool for free! Download, use, fork and extend the OnSite SEO tool from GitHub or visit the demo page.

We wish you a SEO Christmas and a happy new year 2014!

Massive Graph Insert with OrientDB

In our product REWOO Scope we use a postgres database as underlying data storage. While postgres serves a good overall performance, some of our data structure are more graphish like data structures. A standard RDBS does not suit well here – even a recursive SQL query in postgres. Our current implementation needs over 2 1/2 minutes to collect over 70000 vertices of one particular sub graph.  So we investigate some time to evaluate OrientDB, an awesome open source graph-document database.

One of our problem was to load our graph structure with over 1,3M vertices and almost 2,5M edges into OrientDB. The main problem was that the insert of vertices went well, but the insert the edges was a pain. Further we got OConcurrentModificationException here or OutOfMemoryError (perm space) errors there. After some implementation iteration we found a good way:

  • Create indexes before inserting vertices or edges
  • Insert the vertices with OIntentMassiveInsert()
  • Insert the edges with disabled Level 1 cache

Our setting

  • Directed graph with 1,3M vertices and 2,5M edges
  • Each vertex has 3 properties (1x String, 2x Long). Edges do not have properties.
  • Test machine: i7 8×3,4GHz, 8GB RAM, SSD, Ubuntu 13.04, 64 Bit
  • OrientDB 1.5.0, Java 1.7.0_09 with -Xmx6g

Now it needs about 100 seconds to read 1,3M vertices (0.07ms/vertex) and about 370 seconds to read about 2,5M edges (0.15ms/edge). The graph database needs about 450 MB disk space. BTW: the final graph traversal of 70000 vertices took about 4 seconds which is a very good result against our current implementation.

Resources:

For dummy source code read the full blog entry.

Continue reading

SeaCon 2013 – Software Architecture, Processes and Management

I attended the SeaCon 2013 in Hamburg, Germany, a conference about software architecture, processes and management. It was a very delightful conference with lots of fresh talks, good food and awesome gimmicks. Here are my findings of these two days:

  • Evaluate the needs of outsourcing well! In most cases it makes more sense to develop a mobile app or an e-commerce software on your own than by a 3rd party. Knowledge is power. If it is your team that gets into it then you are in control of it. Your team has a strong relationship to the project and you are able to change everything whenever it is required. If you outsource something, make sure that all the software sources and rights belong to you after the external project has finished. Otherwise you will have a kind of vendor lock and have to pay for subsequent changes (which might take ages to complete, too).
  • It is very popular to develop software using agile methods like Scrum, Kanban or Scrum Ban. Today, every modern development is agile. However, the old waterfall software development architecture still applies to everything but the development itself: the management, budgeting and external agreements do not fit into the agile development framework — yet. There is a need to change that. The management should adapt agile methods for quicker decision cycles. The budgeting should be approved and reconsidered in shorter intervals than one year (AKA beyond budgeting in Germany) and contracts to 3rd parties should be more loosened.
  • Similar to the previous topic but different: Large projects cannot be controlled but guided. It is impossible to accurately estimate all aspects of a larger project like its complete feature set, required time, and total costs. However with shorter control and decision cycles you can guide the project better than in one large phase. A decision cycle should not be longer than three months and decisions should be made by a heterogeneous group of specialists.
  • It is more important to consider the business and investment value a specific task has for your company rather than asking the software development department about the required time to complete the job. If you know your business value well, you can use and organize your resources (man power, time, and money) in a better way.

Why findAll on null returns empty lists in Groovy

Recently while debugging our grails application I saw something like this:

variable.findAll { it.condition }

Since the debugger told me that this variable was null, I happily thought I might have found my little bug and moved on waiting for the big crash and a nice NullPointerException. Instead I got an empty list.

Wait WAT?

That seemed rather bizarre to me, so I checked it inside a groovy shell.

null.findAll { true } ==> []
null.findAll { false } ==> []
null.findAll { it } ==> []
null.findAll {} ==> []
null.findAll() ==> []

Ok, what’s going on here? First thing to note is that Groovie’s null is an object.

null.getClass().name ==> org.codehaus.groovy.runtime.NullObject

From the Java perspective kind of surprising but since I’m familiar with Ruby it was somehow expectable. So far no magic, but where does that findAll method come from?

null.getMetaClass().methods*.name
==> [equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait, getMetaClass, getProperty, invokeMethod, setMetaClass, setProperty, asBoolean, asType, clone, equals, getNullObject, getProperty, hashCode, invokeMethod, is, iterator, plus, setProperty, toString]

Not there… so Groovy-Voodoo. Luckily there are some developers more experienced with Groovy than me in our company (even one who contributed to it some time ago), so I could ask someone else than Google. We got the source code (Groovy 1.8 in our case) and dug into it. The place where a lot of those magical methods dwell is DefaultGroovyMethods.java. According to the documentation the static methods in this class will be available in the class of each method’s first parameter. So here we found the following:

public static Collection findAll(Object self, Closure closure) {
    List answer = new ArrayList();
    Iterator iter = InvokerHelper.asIterator(self);
    return findAll(closure, answer, iter);
}

Which at least explains why that is available for null. Furthermore it shows us that findAll should work on any Object, too. A quick check in the console confirms this.

new Object().findAll { true } ==> [java.lang.Object@79ad86e9]

However it does not explain how the invocation works and why the result is []. So what’s happening here? The asIterator method simply invokes a method named iterator on self. Groovie’s NullObject defines this particular method in the following way:

public Iterator iterator() {
    return Collections.EMPTY_LIST.iterator();
}

This clearly explains why we get an empty list from our findAll call. In the case of an arbitrary GroovyObject we again find (after an unsuccessful lookup in GroovyObject.java) the iterator method for objects in the DefaultGroovyMethods class simply putting the object into a collection and iterating over it.

public static Iterator iterator(Object o) {
    return DefaultTypeTransformation.asCollection(o).iterator();
}

What is still missing to a full understanding of this phenomenon is how those default groovy methods get invoked. Covering this would be way beyond the scope of this blog post. If you browse around a little in the source all this meta stuff can get kind of overwhelming. What we can take out of this so far (beside getting confused by Groovie’s method invocation mechanisms) is some more awareness to the fact that anything and everything can happen in languages such as Groovy even when it all starts with an innocent null object…