Wednesday, August 31, 2011

PAW - I18N & Native Mapper

It might seem that there is no process concerning PAW, but actually I'm quite busy :)
My plans are to release a 1.0 version this year. In October PAW will be two years under development and I think it's time to leave beta.

Things I'm currently working on are internationalization and what I call Native Mapping.

Internationalization
Currently I'm translating the web interface into German. This will result in a Java properties file with key and value pairs.
After this is finished it should be quite easy to make a translation for any language.
Translation takes much longer than I thought and I'm sure there will be a lot of mistakes in the first version.

Native Mapper
The PAW BeanShell approach has the advantage of being very flexible but when it comes to performance it's really slow.
As I showed in a previous blog post BeanShell is considerably slower than the same native code.
The Native Mapper is an approach to solve this dilemma.

The Mapper is comparable to a Servlet mapping in a JEE environment. The Native Mapper is implemented as Brazil Handler that sits in front of the BeanShell Handler and redirects URLs to native code (DEXed classes).

Handler Chain

The next version of PAW will use the mapper to speed up the generation of graphics inside the web application. This includes the generation of icons, contact images, album covers and the load graph.
Because some versions of HoneyComb have the problem that the DEX classloader is not working the original BeanShell code is also available. So if the Native Mapper fails to initialize (because the DEX classes are not available), the old BeanShell code is used.

A good example to show the possible speed-up when the Native Mapper is used is the Installed Apps page of the web application. On my Nexus One it shows 131 icons. Without Native Mapper this takes 90 seconds, with Native Mapper only 10.

Speed Test (time in seconds)

I hope to release the new version in the next couple of weeks ... stay tuned :)

Monday, July 18, 2011

DavDrive - Litmus Compliance Test

DavDrive works reasonably well with Linux, Mac and Windows clients. To make DavDrive more RFC 2518 compliant the Litmus compliance test was used.
Litmus contains a variety of tests. Because DavDrive implements only a subset of the RFC specification only three tests are relevant: basic, copymove and http

Litmus Test Results (left DavDrive 1.40, right DavDrive 1.41)
The upcoming DavDrive version 1.41 passes all three tests without failure.
This involved a significant code change, so intensive testing is necessary before publishing the new version.

Tuesday, July 12, 2011

PAW - Dynamic Handlers and Filters

In addition to dynamically loading DEXed classes the latest PAW version provides the possibility to use external so called dynamic Handlers and Filters.

Handlers and Filters are components of the underlying Brazil framework. Handlers are responsible for handling HTTP request. Filters in addition can process the output provided by an handler. In principle PAW consists of a number of handlers that are called in a row until a Handler feels responsible to handle the request.
There is a special Handler, the so called FilterHandler. This Handler can wrap a Handler and directs the output of that Handler to a list of Filters which process the response before it is provided to a Web Browser.

I'll demonstrate this on a small but useful filter that stamps all images inside a defined directory with the PAW logo. Such a technique is often used on sites that provide screenshots and want to automatically decorate them with a logo.

Image without and with Filter applied.

Note:
In this example only the AndroidDynamicFilter is used. There is also  a AndroidDynamicHandler available which can be used if a custom Handler is needed. For this example the AndroidDynamicHandler class is not needed.

Note on Honeycomb: As mentioned in an earlier post, this might not work on Android 3.x because of a bug in Honeycomb.

Prerequisites

What you will need to build the Filter:
Creating a Java Project

When Eclipse is running, we can create a standard Java project.
The Java build path should include the brazil.jar (I've renamed it from brazil-2.3.jar to brazil.jar) and the android.jar file. The android.jar files can be found in the installation directory of the Android SDK (platforms/android-*).

Java Build Path

The Filter Class

After creating the project create a source folder (if not available) and create a new package called dextest.filter.
Inside that package we will create the StampFilter class.

The structure should look something like this:

Project Structure
Before creating the StampFilter class just a view words how a filter works.
A Filter has three important methods:

  • init() - This initializes the Filter and is called on Filter startup (when PAW starts). This returns true on success and false on failure.
  • shouldFilter() - Is called on each Handler output and decides weather the response is to be filtered or not. Returns true if the filter() method should be called, false otherwise.
  • filter() - This is the actual filter method. It processes the content form the Handler and returns it.
Below is the code of the Filter class:

package dextest.filter;

import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Properties;

import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Canvas;

import sunlabs.brazil.filter.Filter;
import sunlabs.brazil.server.Request;
import sunlabs.brazil.server.Server;
import sunlabs.brazil.util.http.MimeHeaders;

public class StampFilter implements Filter {
 private String prefix;

 private static final String STAMP_FILE = "stampFile";

 private static final String IMAGE_DIRECTORY_URL = "imageDirctoryUrl";

 private String stampFile;
 private String imageDirectoryUrl;

 public boolean init(Server server, String prefix) {
  this.prefix = prefix;

  Properties props = server.props;

  stampFile = props.getProperty(prefix + STAMP_FILE, null);
  imageDirectoryUrl = props.getProperty(prefix + IMAGE_DIRECTORY_URL, null);

  if (stampFile != null && imageDirectoryUrl != null) {
   if (!new File(stampFile).exists()) {
    server.log(Server.LOG_ERROR, prefix, STAMP_FILE
      + " does not exist!");
    return false;
   }

   if (!new File(imageDirectoryUrl).isDirectory()) {
    server.log(Server.LOG_ERROR, prefix, IMAGE_DIRECTORY_URL
      + " is not a directory!");
    return false;
   }

   return true;
  } else {
   server.log(Server.LOG_ERROR, prefix, "Missing parameter(s)");
   return false;
  }
 }

 public boolean shouldFilter(Request request, MimeHeaders headers) {
  String type = headers.get("Content-Type");
  return type != null && type.startsWith("image/png") && new File(request.url).getParent().equals(imageDirectoryUrl);
 }

 public byte[] filter(Request request, MimeHeaders headers, byte[] content) {
  try {
   return stampImage(content);
  }
  catch(Exception e) {
   request.log(Server.LOG_ERROR, prefix, "Exception: " + e.getMessage());
   return content;
  }
 }

 /**
  * This is the request object before the content was fetched
  */
 public boolean respond(Request request) throws IOException {
  return false;
 }

 byte[] getImageBytes(String image) throws IOException {
  File f = new File(image);
  byte[] bBitmap = new byte[(int) f.length()];
  FileInputStream fis = new FileInputStream(f);
  fis.read(bBitmap);
  fis.close();
  return bBitmap;
 }

 private byte[] stampImage(byte[] imageBytes) throws IOException {
  byte[] stampBytes = getImageBytes(stampFile);
  Bitmap stampBitmap = BitmapFactory.decodeByteArray(stampBytes, 0, stampBytes.length);

  Bitmap imageBitmap = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
  imageBitmap = imageBitmap.copy(imageBitmap.getConfig(), true);

  Canvas canvas = new Canvas(imageBitmap);
  canvas.drawBitmap(stampBitmap, imageBitmap.getWidth() - stampBitmap.getWidth(), imageBitmap.getHeight() - stampBitmap.getHeight(), null);

  ByteArrayOutputStream bos = new ByteArrayOutputStream();
  imageBitmap.compress(Bitmap.CompressFormat.PNG, 0 /*ignored for PNG*/, bos);

  return bos.toByteArray();

 }
} 

In the init() method you can see that the Filter excepts two parameters. One is called stampFile, the other imageDirectoryUrl. The stampFile parameter specifies the file (absolute path) of the image that is placed on top of the original image. The imageDirectoryUrl parameter specifies the URL for which the filter is applied.
The init() method does check if the parameters are present and if they are valid. The parameters are read from the configuration by using props.getProperty(). The prefix is basically the name of the Filter.

The shouldFilter() method checks if the file requested is a PNG and if the file is served from the URL that should be filtered.

Filtering is actually done in the filter() method. This method will not be discussed in detail, because it uses standard Android functionality to do the stamping.

After compiling the Filter, create a JAR file called filterTest.jar.

Creating the DEXed Filter JAR

Now we can use the dx command from the Android SDK to create a DEXed JAR file:
./dx --dex --output=/tmp/filterTest_dex.jar --positions=lines filterTest.jar

Now copy the resulting filterTest_dex.jar JAR file over to your Android device and store ist directly on the SD Card (/sdcard).

Here is the ready to use JAR file:  filterTest_dex.jar

The Stamp Image

In principle you can use any PNG image for stamping the images.
Here is an example:
Store it directly under /sdcard/paw_powered.png.

Modifying the PAW Configuration

The next step is to modify the PAW configuration. The two important files are handler.xml and filter.xml. Both files are located inside the paw/conf directory.
The handler.xml defines the Handles and the filter.xml defines the Filters used by PAW.
We will start with the filter configuration. For that open the filter.xml file and add the following Filter definition inside the <filters> </filters> tags:

<filter status="active" type="custom">
    <name>Stamp Filter</name>
    <description>Stamp Filter</description>
    <removable>true</removable>
    <id>stamp</id>
    <files />
    <params>
      <param name="stamp.class" value="org.paw.filter.AndroidDynamicFilter" />
      <param name="stamp.filterClass" value="dextest.filter.StampFilter" />
      <param name="stamp.filterJars" value="/sdcard/filterTest_dex.jar" />
      <param name="stamp.stampFile" value="/sdcard/paw_powered.png" />
      <param name="stamp.imageDirctoryUrl" value="/stamp" />
    </params>
  </filter>

PAW knows two filter types httpProxy and custom. HttpProxy filters are only important when PAW is used as a proxy server. In our case the StampFilter is a custom Filter.
The parameters define the Filter class to use, the location of the DEX file, the stamp image and the URL (the directory) where the images that should be stamped are located (we will create hat directory below).


Now comes the Handler part. Our new Handler definition will replace the existing File Handler definition and use the defined StampFilter. So edit the file handle.xml and replace the existing File Handler with the following XML code:
<handler status="active">
    <name>File Handler</name>
    <description></description>
    <removable>true</removable>
    <id>filehandlerWrapper</id>
    <files />
    <params>
      <param name="filehandlerWrapper.class" value="sunlabs.brazil.filter.FilterHandler" />
      <param name="filehandlerWrapper.handler" value="filehandler" />
      <param name="filehandlerWrapper.filters" value="stamp" />

      <param name="filehandler.class" value="sunlabs.brazil.server.FileHandler" />
      <param name="filehandler.root" value="/sdcard/paw/html" />
      <param name="filehandler.defaults" value="index.html" />
    </params>
  </handler>

After that restart the PAW app and let's start testing :)

Using the Filter

For testing, let's create the directory for the images that should be stamped as defined in the Filter definition. For that create a directory called /sdcard/paw/html/stamp and place some PNG images inside it. For a test, you can also put some other image files (e.g. JPG) in that directory. These should not be modified.
Now enter the Url http://<ip number>:8080/stamp into the address bar of your browser and select a PNG image.

The resulting image should include the stamp image in the lower right corner.

Monday, July 11, 2011

PAW - Dynamic DEX Class Loading

With PAW developers have the possibility to create dynamic pages using BeanSell scripts.
This is nice because pages can be developed very fast. On the downside BeanShell is not the fastest scripting language on the planet due to its heavy use of Java reflection. This was demonstrated in an earlier blog entry  where BeanShell code is between 10 and 50 times slower than the native equivalent.

One note beforehand:  Dynamic DEX class loading is currently not working on Honeycomb. This is due to a bug in the in the DexClassLaoder. Google has aknowledged the bug. This bug should be fixed in upcoming Honeycomb releases.

PAW now introduces a new directory called webconf/dex wich can contain JAR and APK files in DEX format. On server startup all files which are present in hat directory will be added to a new classloader. This classloader can be retrieved by calling getDexClassloader() on the service object. To make life easier the command useDexClasses() can be used in BeanShell pages.

To  convert an existing JAR file into a DEXed one the following dex command from the Android SDK (normally located in the platform-tools directory) can be used:
./dx --dex --output=output.jar --positions=lines input.jar

Note: In recent versions of the SDK the dx command has been moved to the build-tools/ directory of the SDK.

If you would like to keep the existing classes inside the JAR file, the additional option --keep-classes can be used. With this option the same JAR file can be used on Android or a PC running standard Java.

After the DEXed JAR file has been created it can be copied to the webconf/dex folder of the PAW installation.

Now to use the new classes you can either restart PAW or issue the following  command from the BeanShell Console:
server.props.get("serviceContext").buildDexClassLoader();

If the new classes should be used inside a BeanShell XHTML page the following command has to be executed:
useDexClasses()

That basically tells BeanShell to use the new classloader.

Example

Here is a very simple Java class that sums up two Intergers:
package dextest;

public class Math {
 public static int sum(int i, int j) {
  return i + j;
 }
}

After expopting the class as JAR file it can be converted into DEXed format using the following command (the JAR file dextest.jar is located in the /tmp directory) :
./dx --dex --output=/tmp/dextest_dex.jar --positions=lines dextest.jar

The resulting dextest_dex.jar JAR file is now placed inside the webconf/dex folder and PAW is restarted.

Now we can use the new class within the BeanShell Console like this:
useDexClasses();

import dextest.Math; 

print(Math.sum(1, 3));

Monday, June 6, 2011

Windows Performance and Filesize Limit

This weekend I had the chance to test DavDrive's performance when running Widows 7 (32bit).
I tested with Windows Explorer and BitKinex. The latter gave me a nice performance graph. My test file was a video file that was approx. 900MB in size.
The first things I notices was that Windows 7 seems to have a file size limit when it comes to WebDAV downloads. Therefore I immediately continued to test with BitKinex because I had no clue how to fix the size limit by that time. You'll find the solution for this after the BitKinex test.
BitKiney worked like charm. The graph shows the download speed.

DavDrive - BitKinex Performance


Speed was quite good (1,48 MB/s) and by far not as bad as many users have reported when using DavDrive on Windows.
So if you fix your Internet Explorer settings and the size limit, DavDrive should run fine and with reasonable speed on Windows.

Windows Size Limit
To fix the size limit you have to use the registry editor and edit the following key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters\FileSizeLimitInBytes

Monday, May 30, 2011

DavDrive - Tablet Support

There is a new version of DavDrive (1.39) and the full version got some enhanced tablet support.
The main screen now presents the usual start button on the left and a list that contains the log entries on the right.

DavDrive 1.39


In default mode the list only shows uploads and downloads so not to confuse the normal user too much. You can switch the display by pressing the small arrow in the activities title bar to display all entries.

Log View Selection


When clicking on these entries a dialog box with some more protocol details is displayed. If the file is present it can be opened.

Log Entry Details


The tablet support is not only for Honeycomb but for all others X-large screens. So other tablet users running Android 2.x should also benefit from the changes.

Log Broadcasts
Another new feature is that the log activities are broadcasted, which means that other programs/activities can listen to these broadcasts and react on these events.

For those interested, here are the details.

Intent Action: de.fun2code.android.webdrive.history.intent.add

ExtraTypeDescription
MethodStringMehthod name in uppercase.
TimestampLongTime in millis.
ResourceStringComplete resource (file) name.
SuccessBooleanTrue on success, otherwise false.
CodeIntegerHTTP result code.
MessageStringAdditional message.
TargetStringComplete target resource (file) name.
This is optional and only available when Method is MOVE.

Tuesday, May 17, 2011

Windows 7 WebDAV Performance

This bugged me for a while...
DavDrive's WebDAV performance seems o.k. on almost all platforms, except for Windows 7 (and especially the 64 bit version).
I have no Windows 7 at home, so I could not really test.

Today I stumbled over a blog entry by Chief Oddball which suggests a fix.
To make it short, here is the possible fix (quoted form the blog entry):
  1. In Internet Explorer, open the Tools menu, then click Internet Options.
  2. Select the Connections tab.
  3. Click the LAN Settings button.
  4. Uncheck the “Automatically detect settings” box.
  5. Click OK until you’re out of dialog hell.
I'm really interested if this is working.
Comments welcome!