Image Recognition of Sights in Graz, Austria

I created an image recognition engine for sights in the city Graz (Austria) with Tensorflow and the inception v3 system. This is an already pretrained network of ImageNet data, which allows you to only train the last layer to adjust the network for specific needs. The training process of the final layer will only take a few seconds to minutes.

It's possible to achieve a recognition rate of about 91 percent after 1000 iterations for several sights with only 100 images per sight. The sight used for classification are:

If we classify some self-taken pictures with the trained network, we get those classifications (the correct classification is given in bold):

Image Classification
Sehenswuerdigkeit graz uhrturm treppe uhrturm stairs: 0.979435
island on the mur: 0.00957681
kunsthaus: 0.00697807
uhrturm: 0.00244436
castle eggenberg: 0.00147081
Sehenswuerdigkeit graz uhrturm uhrturm: 0.971426
island on the mur: 0.012622
uhrturm stairs: 0.00470061
castle eggenberg: 0.00410007
kunsthaus: 0.00313921
Sehenswuerdigkeit graz kunsthaus umgebung kunsthaus: 0.925498
island on the mur: 0.0463341
uhrturm stairs: 0.0104462
uhrturm: 0.00841624
landhaushof: 0.00464753
Sehenswuerdigkeit graz kunsthaus umgebung klein kunsthaus: 0.367763
uhrturm: 0.191424
island on the mur: 0.174073
rathaus: 0.136471
uhrturm stairs: 0.112171
Sehenswuerdigkeit graz uhrturm verdeckt uhrturm: 0.967778
kunsthaus: 0.0199656
uhrturm stairs: 0.00514048
town hall: 0.00431914
island on the mur: 0.00123447
Sehenswuerdigkeit graz rathaus umgebung rathaus: 0.541813
kunsthaus: 0.443715
uhrturm: 0.00643118
island on the mur: 0.00610913
landhaushof: 0.00175862
Sehenswuerdigkeit graz rathaus rathaus: 0.990615
castle eggenberg: 0.00487029
landhaushof: 0.00258762
uhrturm: 0.00160854
uhrturm stairs: 0.000216472
Sehenswuerdigkeit graz kunsthaus kunsthaus: 0.998515
castle eggenberg: 0.000498741
uhrturm stairs: 0.000443049
island on the mur: 0.000394137
landhaushof: 6.33227e-05
Sehenswuerdigkeit graz murinsel murinsel: 0.97064
kunsthaus: 0.0219113
uhrturm stairs: 0.0037573
castle eggenberg: 0.00207429
uhrturm: 0.00125379

I want to highlight the correct classification of the small modern art museum with a lot of environment (0.37 on position 1) and the bad recognition of the town hall in between roofs (0.54 on position 1, but followed by modern art museum on position 2 at 0.44). This probably shows, that the models tends to classify roofs as the modern art museum, even though my training pictures also included pictures of the town hall from top.

Thus, it's possible that the Modern Art Museum with a score of 0.37 was not classied as museum because of the museum itself, but maybe just because of the roofs. Alternatively, the mountains in the background might have been detected as Modern Art Museum.

Judging from the training dataset, I'd say that most of the errors in the top-1-classification are caused by the distinction of Landhaushof and Castle Eggenberg. There are some pictures in my dataset, on which both really look similar. Unfortunately, I did not have self-taken pictures of either for testing.