Showing posts with label java. Show all posts
Showing posts with label java. Show all posts

Wednesday, December 26, 2012

Starting JSONBus: towards a replacement for Cordova/PhoneGap

Security warning 2015


The Cordova project has been fixing security issues in its external domain whitelist plugin over time, and has also dealt with security issues in its internal bridging mechanism. It is very strongly advised NOT to release an app with a custom-built web view bridging or domain whitelisting mechanism without proper code review and testing.

Hybrid architecture for mobile apps

Cordova/PhoneGap provides a framework to develop mobile applications in HTML5, Javascript, and CSS with common APIs to access native device functionality on a number of mobile platforms. Native device functionality that is not built into the Cordova/PhoneGap core can be accessed by using & adapting plugins for the desired platforms. The API function calls between Javascript and the native access classes are almost always asynchronous.

AFAIK there are a couple important things at an itty-bitty level:
  • All native access classes actually subclass from a Cordova plugin class; and
  • All function calls and callbacks are sent between JavaScript & native classes in string form. Function call & callback parameters are generally sent as numbers, strings, or JSON objects.
Despite such a simplistic asynchronous API model, Cordova/PhoneGap has become an increasingly complex framework with a very busy forum.

Simplification: a flow-based architecture

Last year I discovered the concept of flow-based programming which seems to have been used & refined over the past 40 years. Flow-based programming (FBP) breaks a large-scale application architecture down to independently threaded components that communicate via data packets, and there are some nice GUI tools to assemble FBP programs. I like the idea very much, however I have two major issues:
  • Each component is expected to be written as an infinite loop that receives its input as blocking reads; and
  • The data packet format & interface is relatively complex in order to accomodate data that is structural in nature.
I am now in the process of developing an idea called "JSON Flow" to design a flow-based architecture based on JSON messages.

UPDATE (June 2015): The "JSON Flow"/"JSON Bus" idea presented here is inspired by but different from FBP since it relies on (asynchronous) two-way message-based communication.

JSON Bus for a simplified asynchronous API

A subset of the "JSON Flow" idea is "JSON Bus" where Javascript code can send an asynchronous API request as a JSON message and receive a response via a callback. Here is a sample JSON Bus message for a DatePicker request:
{
  from: 'datepickerdemo',
  to: 'datepicker',
  data: {
    mode: 'date', 
    date: '12/12/2012/12/12'
  }
}
Here is a sample callback message:
{
  from: 'datepicker',
  to: 'datepickercb',
  data: {
    date: '12/13/2014'
  }
}


JSON Bus sample for Android

A sample of this idea for Android is available in brodyspark / JSONBus-sample-Android (in public domain). The application contains the following tests:
  • Quick JSON echo test;
  • Open a native DatePicker, which is adapted from the Cordova/PhoneGap DatePicker plugin for Android and display the selected date in the WebView.

Here is a quick tour of the existing sample code:
The first version of the JSON Bus sample does not follow the format shown above, however this should be fixed very soon.

This JSON Bus sample demonstrates an ability to send JSON messages in API function calls & callbacks and integrate some native GUI functionality with  application code developed using HTML5 & Javascript technology.

A first JSON Bus sample version for iOS

As a proof-of-concept, I hacked together a sample version in brodyspark / JSONBus-sample-iOS-with-wvjsbridge, using ExampleApp from marcuswestin / WebViewJavascriptBridge. In this sample, the HTML/Javascript sends a request with JSON data to open a native iOS date picker, which will send a result with JSON data when the user has selected the desired date. UPDATED: This sample is in a very crude state and will be replaced by something better in the near future is now abandoned in favor of: https://github.com/brodyspark/JSONBus-sample-iOS (made without WebViewJavascriptBridge and in public domain)

Next steps

Upcoming steps for the Android version include:
  • Fix existing samples to follow the message format given above;
  • A JSON message routing handler in Java, both for function calls from Javascript and for callbacks;
  • JSON message routing handler in Javascript;
  • Addition of a "subscription" model, where Javascript can make a single JSON Bus function call & continue receiving callbacks based on native events;
  • Adapter classes to include & build Cordova/PhoneGap plugin classes, including some built-in functionality, with little modifications if any.

For iOS the sample should really be is now rebuilt without the WebViewJavascriptBridge (see brodyspark / JSONBus-sample-iOS). I found a couple excellent links how to make function calls back and forth between Javascript & Objective-C here and also here. Then the enhancements from the Android version can be applied to the iOS version.

Monday, December 24, 2012

Using SQLCipher for Android with Cordova/PhoneGap

NOTICE (June 2015): These instructions are completely out-of-date, the following Cordova plugin supports sqlcipher out-of-the-box: https://github.com/litehelpers/Cordova-sqlcipher-adapter

The project brodyspark / PhoneGap-SQLitePlugin-Android provides native sqlite database access with an API that is very close to the HTML5 SQL API and can also provide an interface to SQLCipher for Android. Here are the steps to use SQLCipher with a Cordova/PhoneGap project for Android.


Install sqlite plugin for Android


The first step is to install but not build the sqlite plugin for Android. A good starting point is the Cordova/PhoneGap Android example project, found in the lib/android/example subdirectory of Cordova/PhoneGap as downloaded from phonegap.com/download. These steps are also covered in README.md of brodyspark / PhoneGap-SQLitePlugin-Android. Here are the steps in brief to install the sqlite plugin from PhoneGap-SQLitePlugin-Android:

  • Install SQLitePlugin.js from the Android/assets/www subdirectory into assets/www; for example:
$ cp -v ../PhoneGap-SQLitePlugin-Android/Android/assets/www/SQLitePlugin.js assets/www
  • From the Android/src subdirectory copy the tree com/phonegap/plugin/sqlitePlugin with SQLitePlugin.java into src; for example:
$ cp -rv ../PhoneGap-SQLitePlugin-Android/Android/src/com src
  • Add the plugin element to res/xml/config.xml:
--- config.xml.old  2012-07-24 19:44:49.000000000 +0200
+++ res/xml/config.xml  2012-07-24 19:39:43.000000000 +0200
@@ -32,6 +32,7 @@
     <log level="DEBUG"/>
     <preference name="useBrowserHistory" value="false" />
 <plugins>
+    <plugin name="SQLitePlugin" value="com.phonegap.plugin.sqlitePlugin.SQLitePlugin"/>
     <plugin name="App" value="org.apache.cordova.App"/>
     <plugin name="Geolocation" value="org.apache.cordova.GeoBroker"/>
     <plugin name="Device" value="org.apache.cordova.Device"/>


Install prebuilt SQLCipher for Android


  • Download SQLCipher for Android from android-database-sqlcipher / downloads & unzip the package. Note that the package seems to contain an extra __MACOSX subdirectory which can be ignored.
  • Copy the contents of the lib subdirectory from the package into the libs subdirectory except for commons-codec.jar, for example:
$ cp -rv SQLCipher\ for\ Android\ 2.1.1/libs/* libs
$ rm libs/commons-codec.jar
  • Install the ICU file icudt46l.zip from the assets subdirectory in assets, for example:
$ cp SQLCipher\ for\ Android\ 2.1.1/assets/icudt46l.zip assets

Notes:

  • commons-codec.jar is already part of Cordova/PhoneGap
  • SQLCipher for Android attempts to use the system-provided ICU localisation file but it may not included with all target systems.
  • The x86 subdirectory is only necessary when using x86 as a target; for example, some developers prefer to use an x86 Android simulation platform.

Patches to sqlite plugin to use SQLCipher


In src/com/phonegap/plugin/sqlitePlugin/SQLitePlugin.java make the following changes:

  • In the imports:
@@ -24,7 +24,7 @@
 
 import android.database.Cursor;
 
-import android.database.sqlite.*;
+import net.sqlcipher.database.*;
 
 import android.util.Log;
 
  • In SQLitePlugin.execute():
@@ -63,7 +63,8 @@
     JSONObject o = args.getJSONObject(0);
     String dbname = o.getString("name");
 
-    this.openDatabase(dbname, null);
+    String key = o.getString("key");
+    this.openDatabase(dbname, key);
    }
    else if (action.equals("close")) {
     this.closeDatabase(args.getString(0));
  • In SQLitePlugin.openDatabase():
@@ -152,13 +153,15 @@
   */
  private void openDatabase(String dbname, String password)
  {
+  SQLiteDatabase.loadLibs(this.cordova.getActivity());
+
   if (this.getDatabase(dbname) != null) this.closeDatabase(dbname);
 
   File dbfile = this.cordova.getActivity().getDatabasePath(dbname + ".db");
 
   Log.v("info", "Open sqlite db: " + dbfile.getAbsolutePath());
 
-  SQLiteDatabase mydb = SQLiteDatabase.openOrCreateDatabase(dbfile, null);
+  SQLiteDatabase mydb = SQLiteDatabase.openOrCreateDatabase(dbfile, password, null);
 
   dbmap.put(dbname, mydb);
  }
  • In SQLitePlugin.results2string():
@@ -348,6 +351,7 @@
      for (int i = 0; i < colCount; ++i) {
       key = cur.getColumnName(i);
 
+      /**
       // for old Android SDK remove lines from HERE:
       if(android.os.Build.VERSION.SDK_INT >= 11)
       {
@@ -371,6 +375,7 @@
        }
       }
       else // to HERE.
+      **/
       {
        row.put(key, cur.getString(i));
       }

Build and test


Make sure the project is updated for a target platform:

$ android update project --path . --target <id>


Add a small test program to assets/www/index.html, for example:

--- assets/www/index.html.orig 2012-12-24 22:57:45.000000000 +0100
+++ assets/www/index.html 2012-12-24 23:05:40.000000000 +0100
@@ -34,9 +34,35 @@
             </div>
         </div>
         <script type="text/javascript" src="cordova-2.2.0.js"></script>
-        <script type="text/javascript" src="js/index.js"></script>
+        <script type="text/javascript" src="SQLitePlugin.js"></script>
         <script type="text/javascript">
-            app.initialize();
+        document.addEventListener("deviceready", onDeviceReady, false);
+
+        // Cordova is ready
+        function onDeviceReady() {
+          var db = window.sqlitePlugin.openDatabase({name: "DB", key: "secret1"});
+
+          db.transaction(function(tx) {
+            tx.executeSql('DROP TABLE IF EXISTS test_table');
+            tx.executeSql('CREATE TABLE IF NOT EXISTS test_table (id integer primary key, data text, data_num integer)');
+
+            tx.executeSql("INSERT INTO test_table (data, data_num) VALUES (?,?)", ["test", 100], function(tx, res) {
+              console.log("insertId: " + res.insertId + " -- probably 1");
+              alert("insertId: " + res.insertId + " -- probably 1");
+
+              db.transaction(function(tx) {
+                tx.executeSql("select count(id) as cnt from test_table;", [], function(tx, res) {
+                  console.log("res.rows.length: " + res.rows.length + " -- should be 1");
+                  console.log("res.rows.item(0).cnt: " + res.rows.item(0).cnt + " -- should be 1");
+                  alert("res.rows.item(0).cnt: " + res.rows.item(0).cnt + " -- should be 1");
+                });
+              });
+
+            }, function(e) {
+              console.log("ERROR: " + e.message);
+            });
+          });
+        }
         </script>
     </body>
 </html>



Try a debug install:

$ ant debug install


and the test should run.

To test that the encryption is working, try changing the password key in assets/www/index.html:

--- assets/www/index.html 2012-12-24 23:07:43.000000000 +0100
+++ assets/www/index.html.new 2012-12-24 23:13:38.000000000 +0100
@@ -40,7 +40,7 @@
 
         // Cordova is ready
         function onDeviceReady() {
-          var db = window.sqlitePlugin.openDatabase({name: "DB", key: "secret1"});
+          var db = window.sqlitePlugin.openDatabase({name: "DB", key: "secret2"});
 
           db.transaction(function(tx) {
             tx.executeSql('DROP TABLE IF EXISTS test_table');

and try installing and running again. The program should not work if a different encryption key is used to open the database.

Using cursor type enhancements


As described in this recent posting, some enhancements were made in brodyspark / sqlcipher-android-database to use the actual row/column data type information instead of treating all row/column data as strings. To use these changes:
Now as a test, revert the changes to SQLitePlugin.results2string() then compile & test but first uninstall the old version from the simulator or device.

Sunday, December 23, 2012

Enhancements to SQLCipher db classes for Android

NOTICE (June 2015): These instructions are completely out-of-date, the following Cordova plugin supports sqlcipher out-of-the-box: https://github.com/litehelpers/Cordova-sqlcipher-adapter

SQLCipher provides a special, modified version of SQLite to store data in encrypted form using the Native Development Kit (NDK) on the Android platform. How to rebuild SQLCipher for Android was covered in a recent posting. Unfortunately, SQLCipher for Android is based on an old version of the Android database API and is missing a couple important enhancements:
  • get the data type of each row & column in the results for a SQL query, using Cursor.getType()
  • number of rows affected by a SQL UPDATE or DELETE statement, using SQLiteStatement.executeUpdateDelete()
These enhancements are now integrated in brodyspark / sqlcipher-android-database, which was made as a fork of sqlcipher / android-database-sqlcipher.

These enhancements are especially important for the Cordova/PhoneGap sqlite plugin for Android, in order to properly follow the requirements of the Web SQL API.

The enhancements were obtained from the Android SDK 11 (Honeycomb) version. Only some changes to the cursor classes, database utilities, and SQLiteStatement classes were necessary to get these enhancements working with SQLCipher.

Also the brodyspark / sqlcipher-android-tests fork was made to test these changes.

To compile and test these changes please see my previous posting but with the following changes:

  • For the API: $ git clone git://github.com/brodyspark/sqlcipher-android-database.git
  • Test project: $ git clone git://github.com/brodyspark/sqlcipher-android-tests.git
  • The Makefile is already adapted to work with OSX Homebrew.
NOTE: it is best to use API 11 or greater to compile the code, however it should not matter when running the code. Unfortunately, it still has some problems on API 17, to be fixed sometime in the future.

UPDATES January 2013:

Wednesday, December 19, 2012

Developing Cordova/PhoneGap & other Android apps from the shell

The Homebrew package manager makes it much easier for OSX developers to install the Android SDK tools they need to develop and test Android apps from the command line. Unfortunately, while efforts had been made to port Homebrew to Linux, the benefits of Homebrew for the Android SDK tools have not yet been extended to the Linux platform.

Background: OSX with Homebrew

Here are the steps for using Homebrew to develop Android apps on the command line:

  • install the Apple OSX Command Line Tools from Xcode or by downloading the package from Apple;
  • install Homebrew as described on its homepage;
  • $ brew install android-sdk
  • $ android to open the GUI, install toolkits & desired API versions, and create & run an AVD (Anrdroid Virtual Device)
  • $ adb devices to list the device(s), simulated or real, that are up & running
  • $ android list to get the list of available API targets
  • android create project --package com.example.helloandroid --activity HelloAndroid --target <target-id> --path HelloAndroid where <target-id> is a valid id from android list
  • $ ant debug install to install on the simulator
  • To try the Cordova/PhoneGap Android example: in the project do $ android update project --path $(pwd) --target <target-id> then $ ant debug install



Using the Android SDK with Linux: Ubuntu


Get the Linux version of the Android SDK from the developer.android.com or use a command like the following:
wget http://dl.google.com/android/android-sdk_r21-linux.tgz

Extract using a command like
tar xzvf android-sdk_r21-linux.tgz

Add to .bashrc:
PATH=$HOME/android-sdk-linux/tools:$HOME/android-sdk-linux/platform-tools:$PATH

and refresh with a command like: $ . .bashrc


IMPORTANT: The Android SDK has its dependencies on 32-bit (i686) libraries. For a 64-bit system you will need to do something like:
sudo apt-get install ia32-libs (for more details see this article)


To install JRE, JDK, and ant:

sudo apt-get install openjdk-6-jre
$ sudo apt-get install openjdk-6-jdk
$ sudo apt-get install ant

NOTE: while these are clearly linked by dependencies, it is best to install them one-by-one to make sure the correct versions of all components are installed. I first tried using ant to install all JRE/JDK components and got a Headless AWT exception.

To run the Android GUI:
$ android

Select & get the SDK platform tools, a recent version of the SDK platform, and an ARM system image from a recent SDK platform. Unfortunately the GUI does not seem to download very quickly so only start with the components you really need.

Once a recent version of the SDK platform & system image are downloaded and installed, open the Tools menu & select Manage AVDs. Create a new AVD with some reasonable values and try to start one.

If it does not start, here are some things to check:

  • verify that there is a valid JRE with a real GUI;
  • double-check that the correct PATH has been set & refreshed;
  • make sure the installation of the SDK & platform tools is ok.

List the valid API target IDs:
$ android list

Create a test project from the command line:
android create project --package com.example.helloandroid --activity HelloAndroid --target <target-id> --path HelloAndroid where <target-id> is a valid ID from $ android list

In the project directory, try to build & install:
$ ant debug install

If the installation goes well, the app should display a welcome message when you open it.

To watch the log events: $ adb logcat

To try a Cordova/PhoneGap project, in the example from lib/android:
android update project --path $(pwd) --target 1
and then try $ ant debug install


Notes for using the Android SDK with Fedora Core


The procedure to install and use the Android SDK is very similar between Ubuntu & Fedora Core. The major differences lie in the system dependencies.

The JRE, JDK, and ant should be installed with a different procedure than for Ubuntu. From this article the JRE & JDK can be installed with a command like:
yum install java-*-openjdk java-*-openjdk-plugin

For a 64-bit system, please make sure the following i686 packages are installed: glibc.i686libstdc++.i686ncurses-libs.i686zcore.i686, and zlib.i686.

In the future, I would like to download, install, and update the Android SDK & tools using Homebrew on Linux as well.