Qt on iOS and Android. Looking native today. Or maybe tomorrow

That’s a long post about what could be the ways for making a native looking and feeling, and behaving Qt app for iOS and Android, so here’s a table of contents:

Qt and mobile apps

Coming from strong Nokia background I like Qt and mobiles. I used to be a Forum Nokia Champion, Qt Ambassador, created a number of Qt/QML apps for Nokia N9 and Symbian, possibly created the first ever 3rd party app available from MeeGo, Symbian and Android app stores and built from the same source code. Nowadays I am spitting out Jolla Sailfish apps (including the app more popular than the system Clock app) with the machine gun speed just because I love to see power of Qt and QML come to action on the fully-Qt device.

I am also a UI standards maniac (not necessarily a succeeding one) and it drives me crazy that I cannot use my favorite Qt and QML for building mobile apps for mass market (read: Android and iOS), now that it is technically possible, but in practice only low level graphics examples exist and nobody seems to care much about native or native-looking toolbars, gestures, sharing dialogs, integrating to notification services, you name it.

Right now I am flying for a New York reunion of my day job company (BTW, have a look at pretty cool interactive images service we make, sure we have mobile apps for creating these on the go too) and drafting this post for outlining a set of experiments to proof or decline the ability to create native-looking iOS and Android apps already today or tomorrow. I will probably try implementing a couple of these and some will be beyond my skills. So if you are into the same stuff, grab a topic, do an example on github and post it to comments, I’ll add it to the post.

What’s needed for app to be native-enough

So what’s needed for an app to be native enough. Two things: native look&feel and integration to the system services.

Native look and feel

Buttons, toolbars, gestures (e.g. iOS 7 Back gesture), native text input, view transitions, theming according to the current device theming. Sure it’s better for these to be fully native, but looking at what modern mobile HTML5 app frameworks are able to achieve, I think it’s okay at least for now if your app will look almost native. For example, if you implement view transitions yourself so that they look almost native and make buttons that look like the default phone theme only, I won’t be ashamed for shipping such an app to app store. As a rule of thumb: if you would feel okay showing the next Instagram-Twitter-Foursquare app done this way to potential investors, I’d say UI is looking native enough.

Integration to the system services

There are non-visual or partially non-visual services that user got to expect from the native apps nowadays (and not immediately supported by Qt). Two prime examples are system sharing sheets and system accounts integration (logging in/signing up with Facebook, Twitter, Google+). As the secondary examples I would add being on the receiving side of the sharing interaction and working in the background when started by the system notifications service or just because your app is registered for the background execution. Sure you could add many more examples, but if the ones above are covered, everything else should be doable to and you will probably be able to modify the existing proof-of-concept demos for it.

What’s possible for making the native-enough app

Native look and feel

There is a number of approaches available:

Plain Qt Quick

Plain Qt Quick

You get a rectangle and you draw there whatever you like. That is the way the current most cool mobile Qt demos are made, performance seems to be good enough for cool effects and fast animation on all the relatively modern devices. That could be a good approach for creating a cross-platform mobile game nowadays where you anyway are likely to have non-system looking controls and not much system interaction.. except for in-app purchases.

  • Pros: You can use power of the full Qt and QML, manipulate your C++ objects and QML and create whichever effects you want easily
  • Cons: If you are not creating just a demo game, cloning system toolbars and view transitions could be a daunting and boring task. Possibly an easy one, possibly a complex one

Qt Quick Controls

Qt quick controls

That is the primary way forward for Qt for Android. There is an effort led by Bogdan Vatra for making existing Qt Quick controls look real native and even fetching colors and font sizes from the system theme. The problem is that most of these goodies are [probably] coming in the end of 2014 the earliest and are for Android only anyway. Meanwhile you can grab the current, quite desktop-looking controls and with a bit of hammer and swearing make them look good enough for you. There is even a couple of open source projects aimed at it.

  • Pros: Full Qt/QML power is still here, you will definitely find at least PageStack and Page elements useful, possibly some controls won’t look that bad.
  • Cons: You won’t get theming for free, possibly making even semi-okay looking controls will be difficult, native looking transitions and gestures will also be on you

HTML5 with C++ interfaces

HTML with large QML part

Qt 5.2 SDK already includes an HTML5 app template and, well, it is pretty much just a Webkit view with HTML inside, so you can use some modern mobile app frameworks for the native-looking buttons and toolbars. Advantage you get from using these frameworks in the HTML only mode is [potentially] easy integration to the C++ side on your code. I think you could access C++ side objects injected to HTML or made accessible by it (e.g. Settings or file system or your proven network comms engine). And/or you can go the other way around and modify web page DOM elements from C++.

  • Pros: Native-enough look nearly for free. You can still use your Qt libraries and shift much of the complexity to the Qt side
  • Cons: No QML, Qt-HTML integrations could be difficult and unusual (e.g. could signals and slots be used anyhow?), no good debugging and profiling tools

HTML5 with the large part painted in Qt/QML

HTML with large QML part

A variation of the above for the apps centered around large main controls such as lists, map of the local restaurants or a painting board. It is likely to be possible to paint these using Qt/QML though performance and ease of debugging of such an option are to be studied yet. Controling lifetime of such objects could also be tough.

  • Pros: You get native enough-looking controls nearly for free while using Qt/QML exactly where it matters most and where you want to polish the UI.
  • Cons: Debugging and optimization could be difficult, running two JavaScript interpreters and passing data/signals back and forth through several layers (or some side bus) could be challenging

HTML5 inside QML

HTML5 inside QML

That is sort of a reverse approach. You can have a QML app that devotes 100% of it’s space to a web view or you can even have a non-visual QML component running side by side with the web view instantiated the C++ way. Then you can paint 100% of UI using HTML5 UI frameworks while having app logic in QML. If you go one step further, you can even create a whole presentation layer in QML that just maps itself into as lightweight as possible HTML5 UI.

  • Pros: Native enough-looking controls from HTML5, all the logic in Qt/QML, property bindings included
  • Cons: It could be tough to establish one-two one mapping between QML and HTML5 when reading and writing data

QML inside HTML5 inside QML

QML inside HTML5 inside QML

I.e. main UI list done in QML injected into web view created by a QML app. Just because we can 🙂
I am far from being sure it is easy to do, but if it happens to be easy, you could get the best of both worlds: controls and view interaction only from HTML5 frameworks, power of QML logic and graphics for everything else.

  • Pros: HTML5 only where it’s really needed (just for native looking controls/transitions) and nothing else. Everything else in Qt/QML
  • Cons: Could be too tough to do that many levels of injection

Real native manipulated by QML

Real native manipulated by QML

QML is not the only Qt way for contracting UI. You can still use a form designer for getting a QtWidgets app and manipulate the generated form from C++ (technically from QML too though you are not likely to do it).

Nowadays both Android and iOS have a way to define UI declaratively and certainly you can use native code for adding more controls. Well, you could define such layouts in the native tools and then instantiate and manipulate them from Qt/QML, possibly even painting a couple of particular controls yourself.

  • Pros: Native look and feel, no compromises whosoever!
  • Cons: Quite platform-specific way for integration, injecting your own controls inside can be tough

Integration to the system services

That is the area I am not really familiar with unfortunately so can’t outline many approaches. I only know that it should be possible to communicate with the Android side via JNI and some way for accessing iOS APIs should also be possible. How easy or hard it is and whether you will be able to use e.g. platform-specific Facebook SDK for logging people in is to be studied.

Experiments to perform

So here is a list of proof of concept demos I think are needed for validating the approaches outlined. Once I implement some of them or figure out an existing demo about the same, I’ll add the links to here. Demo projects could about whatever topic, I guess a basic Flickr browser could make a good reference app.

  1. Looks and feel
    1. Plain QML or plain QML + just PageStack from Quick controls. Just make them look native(ish)
    2. QtQuick controls styles to look like iOS/Android
    3. QtQuick controls styles to look like iOS/Android
    4. HTML5 (with some mobile UI framework) + interaction with the Qt objects
    5. HTML5 with some area inside painted by Qt/QML, ideally a couple of different controls in a couple of different pages
    6. HTML5 inside QML app and manipulated from QML
      • And QML inside HTML5 inside QML
    7. Native iOS layout(s) manipulated from Qt
    8. Native Android layout(s) manipulated from Qt
  2. System integration
    1. Logging in with the system integrated Facebook/Twitter/Google+ (probably many different platform-specific projects)
    2. Passing link/photo to Android sharing framework
    3. Passing link/photo to iOS sharing framework
    4. Receiving link/photo on Android from sharing on Android
    5. Receiving link/photo on iOS. Okay, that’s not very important as I think on iOS the sharing source has to exactly know your app for sharing to you
    6. Sharing link/photo using exactly Facebook/Twitter/G+ SDKs for fully optimized native look
    7. Waking up on system notifications
    8. Waking up on geofencing event or something like that
    9. Just running in the background sometimes

Your opinion

What do you think? I created this list first of all for my own understanding. I am likely to actually try a couple of approaches, but will probably stop once I figure a good enough solution for some real app to make or if I give up due to the complexity. Then this proof-of-concept catalog could continue only if there’s an active interest from somebody to make consumer grade Qt apps a reality on iOS and Android.

Does this list of approaches make any sense to you?
Do you know of some existing projects already trying these? Is something important missing?

Do you think the whole goal of making cool looking Qt apps for iOS and Android is worth studying at all while there are good 100% native platform-specific tools already and even HTML5 for those into the cross-platform UI dream?

  • KonaCode

    Great in-depth analysis from a software architecture standpoint. In your section titled “experiments to perform -> looks and feel”, entries 2 and 3 are duplicate.

    I like the HTML5 inside of QML approach and believe this should be the way forward. In my opinion, abstracting the presentation layer and shifting responsibility to the toolkit is a good thing. At the same time allowing a compiled set of logic is an acceptable native component. This arrangement resembles an MVC architecture and is most similar to today’s modern web apps. I am investigating the data I/O as you described. The con you mention for this approach may not be a valid concern, though, as Qt themselves should ensure that data can be managed appropriately between the two.

    With the above architecture, you would develop for each platform using Qt presentation capabilities as intended, yet design UI that would give a consistent look and feel across all platforms. That to me seems like a step in the right direction to bring existing and future devices closer to each other in familiarity with HTML5. I would want my software to look the same on a desktop, tablet, or mobile and HTML5/CSS3 as a design language offers just that.

    • artemmarchenko

      Indeed, KonaCode, entries 2 and 3 are duplicates, not sure what I actually wanted to be there.. will think. Or will just delete the duplicate line.

      Meanwhile I had a look at what happens in Qt Controls for Android. Qt 5.4 will have a pretty good set of controls (with the notable exception of native web view) with the style fetched from the OS. Since that seems to be become a QtProject-supported way, promises good results and there is no native web view anyway yet, I am leaning towards it.

      iOS is even more difficult web-wise as non-standard web views are simply prohibited so it seems HTML5 inside QML right now is simply impossible (could be possible when native webview is implemented), so for iOS I started a project for simulating a real app UI in QML already now with the aim of, well, validating QML prototyping capabilities (I would like to open business around it) and of figuring out the bottlenecks in controls styling – http://www.codingsubmarine.com/cutecontacts-ios7-cloning-ios7-contacts-application-ui-in-qtqml/

      You are certainly welcome to join CuteContacts and to post links to your experiments in the area, I’ll add links to this post then.

  • Matthew Blott

    Interesting points, well done for putting this together. I initially looked at Kivy as a mobile solution but moved away because it’s only being used for games and the UI is somewhat lacking. I’d also really like a full native UI for my apps which is why I looked at Qt but I soon found out this isn’t available at the moment. I’m now looking at Rubymotion which will soon have Android support and allows you to write fully native apps.

    • artemmarchenko

      I did an experiment with implementing iOS Contacts app in QML and.. it is tough http://www.codingsubmarine.com/cute-contacts-for-ios-video-demo/

      IMHO nowadays you need to be a real big fan of Qt (and probably work on a noncommercial project) to build UI in Qt unless you are creating everything from scratch anyway (e.g. games could be fine). The workarounds in this blog post are possible and will let you use QML, but.. is QML that good to suffer from side effects? IMHO, waiting until proper controls and QtProject-backed extensions is the most sensible way for “just mobile app developers” for now.

  • Pingback: Porting Qt apps from desktop to mobile: look and feel: developer responsibilities | plashless

  • Artem

    I don’t uderstand about using Qt WebView in “HTML5 with C++ interfaces” and “HTML5 with the large part painted in Qt/QML”. Qt does not support Qt WebView on Android and iOS (http://qt-project.org/qt5/qt52). Could you explain this, please?

    • artemmarchenko

      Qt WebView to my understanding is supported on Android since Qt 5.4 or is going to be supported some time soon. That won’t work for iOS though indeed, on iOS you have to use standard iOS WebView and then this approach won’t work. Except you can probably paint over such WebView, but that gets complicated and brittle.

  • Attila Csipa

    I’ve been playing with the topic a bit (not a fan of lowest-common-denominator approaches) and my favorite is platform-native + QML. It seems to be even reasonably simple to do, at least on Android and desktop, I even cobbled together a proof-of-concept: http://achipa.blogspot.com/2014/11/qml-wrappers-for-native-android.html

    • artemmarchenko

      You did awesome work there, Attila! I should comment with more detail on your blog. Shortly, I think it could work for using QML for Android/iOS apps, but you will have to know platform specifics and then.. what so big benefit QML gives you? Just property binding and JavaScript? Good stuff, but is it good enough at a cost of the need to create wrappers for pretty much everything solve threading issues you mentioned and messing with platform stuff anyway?

      • Shadi Saleh

        Hello
        I try to build Notifications app for (Andriod ,ios ,windows phone) the functionality for
        this app is to receive the message from c++ program which work on server and
        show this message for customer so can you advice me if QT power enough to do
        this task or what your advice

        Thanks a lot
        can send me your advice to my email if it is possible :
        shadimsaleh@gmail.com

  • Shadi Saleh

    Hello
    I try to build Notifications app for (Andriod ,ios ,windows phone) the functionality for
    this app is to receive the message from c++ program which work on server and
    show this message for customer so can you advice me if QT power enough to do
    this task or what your advice

    Thanks a lot
    can send me your advice to my email if it is possible :
    shadimsaleh@gmail.com

    • artemmarchenko

      In general, you can do anything in Qt as long as you don’t care much about the app looking and feeling exactly native.

      In practice though there aren’t many Android/iOS/WP apps built with Qt yet, so you may like to ask for the advice on the Qt-related discussion boards.