Monday, July 18, 2011

DavDrive - Litmus Compliance Test

DavDrive works reasonably well with Linux, Mac and Windows clients. To make DavDrive more RFC 2518 compliant the Litmus compliance test was used.
Litmus contains a variety of tests. Because DavDrive implements only a subset of the RFC specification only three tests are relevant: basic, copymove and http

Litmus Test Results (left DavDrive 1.40, right DavDrive 1.41)
The upcoming DavDrive version 1.41 passes all three tests without failure.
This involved a significant code change, so intensive testing is necessary before publishing the new version.

Tuesday, July 12, 2011

PAW - Dynamic Handlers and Filters

In addition to dynamically loading DEXed classes the latest PAW version provides the possibility to use external so called dynamic Handlers and Filters.

Handlers and Filters are components of the underlying Brazil framework. Handlers are responsible for handling HTTP request. Filters in addition can process the output provided by an handler. In principle PAW consists of a number of handlers that are called in a row until a Handler feels responsible to handle the request.
There is a special Handler, the so called FilterHandler. This Handler can wrap a Handler and directs the output of that Handler to a list of Filters which process the response before it is provided to a Web Browser.

I'll demonstrate this on a small but useful filter that stamps all images inside a defined directory with the PAW logo. Such a technique is often used on sites that provide screenshots and want to automatically decorate them with a logo.

Image without and with Filter applied.

Note:
In this example only the AndroidDynamicFilter is used. There is also  a AndroidDynamicHandler available which can be used if a custom Handler is needed. For this example the AndroidDynamicHandler class is not needed.

Note on Honeycomb: As mentioned in an earlier post, this might not work on Android 3.x because of a bug in Honeycomb.

Prerequisites

What you will need to build the Filter:
Creating a Java Project

When Eclipse is running, we can create a standard Java project.
The Java build path should include the brazil.jar (I've renamed it from brazil-2.3.jar to brazil.jar) and the android.jar file. The android.jar files can be found in the installation directory of the Android SDK (platforms/android-*).

Java Build Path

The Filter Class

After creating the project create a source folder (if not available) and create a new package called dextest.filter.
Inside that package we will create the StampFilter class.

The structure should look something like this:

Project Structure
Before creating the StampFilter class just a view words how a filter works.
A Filter has three important methods:

  • init() - This initializes the Filter and is called on Filter startup (when PAW starts). This returns true on success and false on failure.
  • shouldFilter() - Is called on each Handler output and decides weather the response is to be filtered or not. Returns true if the filter() method should be called, false otherwise.
  • filter() - This is the actual filter method. It processes the content form the Handler and returns it.
Below is the code of the Filter class:

package dextest.filter;

import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Properties;

import android.graphics.Bitmap;
import android.graphics.BitmapFactory;
import android.graphics.Canvas;

import sunlabs.brazil.filter.Filter;
import sunlabs.brazil.server.Request;
import sunlabs.brazil.server.Server;
import sunlabs.brazil.util.http.MimeHeaders;

public class StampFilter implements Filter {
 private String prefix;

 private static final String STAMP_FILE = "stampFile";

 private static final String IMAGE_DIRECTORY_URL = "imageDirctoryUrl";

 private String stampFile;
 private String imageDirectoryUrl;

 public boolean init(Server server, String prefix) {
  this.prefix = prefix;

  Properties props = server.props;

  stampFile = props.getProperty(prefix + STAMP_FILE, null);
  imageDirectoryUrl = props.getProperty(prefix + IMAGE_DIRECTORY_URL, null);

  if (stampFile != null && imageDirectoryUrl != null) {
   if (!new File(stampFile).exists()) {
    server.log(Server.LOG_ERROR, prefix, STAMP_FILE
      + " does not exist!");
    return false;
   }

   if (!new File(imageDirectoryUrl).isDirectory()) {
    server.log(Server.LOG_ERROR, prefix, IMAGE_DIRECTORY_URL
      + " is not a directory!");
    return false;
   }

   return true;
  } else {
   server.log(Server.LOG_ERROR, prefix, "Missing parameter(s)");
   return false;
  }
 }

 public boolean shouldFilter(Request request, MimeHeaders headers) {
  String type = headers.get("Content-Type");
  return type != null && type.startsWith("image/png") && new File(request.url).getParent().equals(imageDirectoryUrl);
 }

 public byte[] filter(Request request, MimeHeaders headers, byte[] content) {
  try {
   return stampImage(content);
  }
  catch(Exception e) {
   request.log(Server.LOG_ERROR, prefix, "Exception: " + e.getMessage());
   return content;
  }
 }

 /**
  * This is the request object before the content was fetched
  */
 public boolean respond(Request request) throws IOException {
  return false;
 }

 byte[] getImageBytes(String image) throws IOException {
  File f = new File(image);
  byte[] bBitmap = new byte[(int) f.length()];
  FileInputStream fis = new FileInputStream(f);
  fis.read(bBitmap);
  fis.close();
  return bBitmap;
 }

 private byte[] stampImage(byte[] imageBytes) throws IOException {
  byte[] stampBytes = getImageBytes(stampFile);
  Bitmap stampBitmap = BitmapFactory.decodeByteArray(stampBytes, 0, stampBytes.length);

  Bitmap imageBitmap = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
  imageBitmap = imageBitmap.copy(imageBitmap.getConfig(), true);

  Canvas canvas = new Canvas(imageBitmap);
  canvas.drawBitmap(stampBitmap, imageBitmap.getWidth() - stampBitmap.getWidth(), imageBitmap.getHeight() - stampBitmap.getHeight(), null);

  ByteArrayOutputStream bos = new ByteArrayOutputStream();
  imageBitmap.compress(Bitmap.CompressFormat.PNG, 0 /*ignored for PNG*/, bos);

  return bos.toByteArray();

 }
} 

In the init() method you can see that the Filter excepts two parameters. One is called stampFile, the other imageDirectoryUrl. The stampFile parameter specifies the file (absolute path) of the image that is placed on top of the original image. The imageDirectoryUrl parameter specifies the URL for which the filter is applied.
The init() method does check if the parameters are present and if they are valid. The parameters are read from the configuration by using props.getProperty(). The prefix is basically the name of the Filter.

The shouldFilter() method checks if the file requested is a PNG and if the file is served from the URL that should be filtered.

Filtering is actually done in the filter() method. This method will not be discussed in detail, because it uses standard Android functionality to do the stamping.

After compiling the Filter, create a JAR file called filterTest.jar.

Creating the DEXed Filter JAR

Now we can use the dx command from the Android SDK to create a DEXed JAR file:
./dx --dex --output=/tmp/filterTest_dex.jar --positions=lines filterTest.jar

Now copy the resulting filterTest_dex.jar JAR file over to your Android device and store ist directly on the SD Card (/sdcard).

Here is the ready to use JAR file:  filterTest_dex.jar

The Stamp Image

In principle you can use any PNG image for stamping the images.
Here is an example:
Store it directly under /sdcard/paw_powered.png.

Modifying the PAW Configuration

The next step is to modify the PAW configuration. The two important files are handler.xml and filter.xml. Both files are located inside the paw/conf directory.
The handler.xml defines the Handles and the filter.xml defines the Filters used by PAW.
We will start with the filter configuration. For that open the filter.xml file and add the following Filter definition inside the <filters> </filters> tags:

<filter status="active" type="custom">
    <name>Stamp Filter</name>
    <description>Stamp Filter</description>
    <removable>true</removable>
    <id>stamp</id>
    <files />
    <params>
      <param name="stamp.class" value="org.paw.filter.AndroidDynamicFilter" />
      <param name="stamp.filterClass" value="dextest.filter.StampFilter" />
      <param name="stamp.filterJars" value="/sdcard/filterTest_dex.jar" />
      <param name="stamp.stampFile" value="/sdcard/paw_powered.png" />
      <param name="stamp.imageDirctoryUrl" value="/stamp" />
    </params>
  </filter>

PAW knows two filter types httpProxy and custom. HttpProxy filters are only important when PAW is used as a proxy server. In our case the StampFilter is a custom Filter.
The parameters define the Filter class to use, the location of the DEX file, the stamp image and the URL (the directory) where the images that should be stamped are located (we will create hat directory below).


Now comes the Handler part. Our new Handler definition will replace the existing File Handler definition and use the defined StampFilter. So edit the file handle.xml and replace the existing File Handler with the following XML code:
<handler status="active">
    <name>File Handler</name>
    <description></description>
    <removable>true</removable>
    <id>filehandlerWrapper</id>
    <files />
    <params>
      <param name="filehandlerWrapper.class" value="sunlabs.brazil.filter.FilterHandler" />
      <param name="filehandlerWrapper.handler" value="filehandler" />
      <param name="filehandlerWrapper.filters" value="stamp" />

      <param name="filehandler.class" value="sunlabs.brazil.server.FileHandler" />
      <param name="filehandler.root" value="/sdcard/paw/html" />
      <param name="filehandler.defaults" value="index.html" />
    </params>
  </handler>

After that restart the PAW app and let's start testing :)

Using the Filter

For testing, let's create the directory for the images that should be stamped as defined in the Filter definition. For that create a directory called /sdcard/paw/html/stamp and place some PNG images inside it. For a test, you can also put some other image files (e.g. JPG) in that directory. These should not be modified.
Now enter the Url http://<ip number>:8080/stamp into the address bar of your browser and select a PNG image.

The resulting image should include the stamp image in the lower right corner.

Monday, July 11, 2011

PAW - Dynamic DEX Class Loading

With PAW developers have the possibility to create dynamic pages using BeanSell scripts.
This is nice because pages can be developed very fast. On the downside BeanShell is not the fastest scripting language on the planet due to its heavy use of Java reflection. This was demonstrated in an earlier blog entry  where BeanShell code is between 10 and 50 times slower than the native equivalent.

One note beforehand:  Dynamic DEX class loading is currently not working on Honeycomb. This is due to a bug in the in the DexClassLaoder. Google has aknowledged the bug. This bug should be fixed in upcoming Honeycomb releases.

PAW now introduces a new directory called webconf/dex wich can contain JAR and APK files in DEX format. On server startup all files which are present in hat directory will be added to a new classloader. This classloader can be retrieved by calling getDexClassloader() on the service object. To make life easier the command useDexClasses() can be used in BeanShell pages.

To  convert an existing JAR file into a DEXed one the following dex command from the Android SDK (normally located in the platform-tools directory) can be used:
./dx --dex --output=output.jar --positions=lines input.jar

Note: In recent versions of the SDK the dx command has been moved to the build-tools/ directory of the SDK.

If you would like to keep the existing classes inside the JAR file, the additional option --keep-classes can be used. With this option the same JAR file can be used on Android or a PC running standard Java.

After the DEXed JAR file has been created it can be copied to the webconf/dex folder of the PAW installation.

Now to use the new classes you can either restart PAW or issue the following  command from the BeanShell Console:
server.props.get("serviceContext").buildDexClassLoader();

If the new classes should be used inside a BeanShell XHTML page the following command has to be executed:
useDexClasses()

That basically tells BeanShell to use the new classloader.

Example

Here is a very simple Java class that sums up two Intergers:
package dextest;

public class Math {
 public static int sum(int i, int j) {
  return i + j;
 }
}

After expopting the class as JAR file it can be converted into DEXed format using the following command (the JAR file dextest.jar is located in the /tmp directory) :
./dx --dex --output=/tmp/dextest_dex.jar --positions=lines dextest.jar

The resulting dextest_dex.jar JAR file is now placed inside the webconf/dex folder and PAW is restarted.

Now we can use the new class within the BeanShell Console like this:
useDexClasses();

import dextest.Math; 

print(Math.sum(1, 3));

Monday, June 6, 2011

Windows Performance and Filesize Limit

This weekend I had the chance to test DavDrive's performance when running Widows 7 (32bit).
I tested with Windows Explorer and BitKinex. The latter gave me a nice performance graph. My test file was a video file that was approx. 900MB in size.
The first things I notices was that Windows 7 seems to have a file size limit when it comes to WebDAV downloads. Therefore I immediately continued to test with BitKinex because I had no clue how to fix the size limit by that time. You'll find the solution for this after the BitKinex test.
BitKiney worked like charm. The graph shows the download speed.

DavDrive - BitKinex Performance


Speed was quite good (1,48 MB/s) and by far not as bad as many users have reported when using DavDrive on Windows.
So if you fix your Internet Explorer settings and the size limit, DavDrive should run fine and with reasonable speed on Windows.

Windows Size Limit
To fix the size limit you have to use the registry editor and edit the following key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WebClient\Parameters\FileSizeLimitInBytes

Monday, May 30, 2011

DavDrive - Tablet Support

There is a new version of DavDrive (1.39) and the full version got some enhanced tablet support.
The main screen now presents the usual start button on the left and a list that contains the log entries on the right.

DavDrive 1.39


In default mode the list only shows uploads and downloads so not to confuse the normal user too much. You can switch the display by pressing the small arrow in the activities title bar to display all entries.

Log View Selection


When clicking on these entries a dialog box with some more protocol details is displayed. If the file is present it can be opened.

Log Entry Details


The tablet support is not only for Honeycomb but for all others X-large screens. So other tablet users running Android 2.x should also benefit from the changes.

Log Broadcasts
Another new feature is that the log activities are broadcasted, which means that other programs/activities can listen to these broadcasts and react on these events.

For those interested, here are the details.

Intent Action: de.fun2code.android.webdrive.history.intent.add

ExtraTypeDescription
MethodStringMehthod name in uppercase.
TimestampLongTime in millis.
ResourceStringComplete resource (file) name.
SuccessBooleanTrue on success, otherwise false.
CodeIntegerHTTP result code.
MessageStringAdditional message.
TargetStringComplete target resource (file) name.
This is optional and only available when Method is MOVE.

Tuesday, May 17, 2011

Windows 7 WebDAV Performance

This bugged me for a while...
DavDrive's WebDAV performance seems o.k. on almost all platforms, except for Windows 7 (and especially the 64 bit version).
I have no Windows 7 at home, so I could not really test.

Today I stumbled over a blog entry by Chief Oddball which suggests a fix.
To make it short, here is the possible fix (quoted form the blog entry):
  1. In Internet Explorer, open the Tools menu, then click Internet Options.
  2. Select the Connections tab.
  3. Click the LAN Settings button.
  4. Uncheck the “Automatically detect settings” box.
  5. Click OK until you’re out of dialog hell.
I'm really interested if this is working.
Comments welcome!

Thursday, April 21, 2011

PAW Server & PHP

In this post I would like to write about what is coming up next in PAW development.
PAW supports BeanShell code to create dynamic pages. BeanShell is well suited for Android, because it integrates will into the OS by supporting the Android Java API.

The foundation of PAW is Brazil a web application framework which was developed by SUN (now Oracle). Although not widely used the framework has the advantage of being very small and flexible.
What I was thinking about for a while was to add PHP support. PHP is widely used  around the net. PHP's reputation is questionable and that are people who love and others who hate it. But it's undoubtedly one of the most used frameworks when it comes to web development.

PHP basically comes in two flawors, a CLI and a CGI version. To integrate PHP into the context of a web server the CGI version is needed.

Android PHP Project
Knowing that there is a CLI version provided by the Android PHP Project I headed over to their forum asking if they could provide a CGI version in addition to the CLI version.

Unfortunately the guys told me that a CGI version will not be provided by the project because it's not needed and pointed me to a page which describes shortly how to setup the tool-chain to do a cross compile of PHP.

Cross Compiling
So I tried to compile PHP myself. I have a Linux box at home so I'm quite familiar with doing configure/make and all this things but the last time I dug into C sources and header files is approx. 10 years back. So this was quite a challenge. Finally I got it working, but I think I did a lousy job. So if somebody can do it better, please go ahead and send me the php-cgi binary.

The CGI Handler
After I had the binary I extended and changed the CGI handler that is included with Brazil to work with Android. Due to some changes in the Brazil source that I have done concerning upload size limits the original handler could not be used. In the process I also tried to add SL4A support. I haven't tested this until know, but I hope it will be working.

SL4A Support
SL4A support is a little bit tricky, because one has to know the port the SL4A is listening to. So I looked around how to get hold of the port and found out that the port is reported in the logcat.
So on startup of the handler (on start of the PAW service) the handler looks into logcat and tries to find the SL4A host name and port number. These values are associated with the AP_HOST and AP_PORT environment variables. So SL4A the server has to be started shortly before PAW starts up.

The Plugin
PHP will be available as plugin shortly after the next release of PAW (which should be in the next month). PAW will then come with the new handler. Providing PHP as plugin has the advantage that PAW itself does not grow (much) in size. PHP is approx. 3 MB in size and so relatively large.
The plugin can be downloaded from an external website and extracted into the PAW plugin folder.
A new page showing available plugins will be included into the next PAW release.

Plugins in the Add-Ons menu


After extracting the plugin it should be visible under PAW's plugin menu.

Plugin in the Plug-ins menu


Installation of PHP should be a one click action and after a restart of PAW PHP should be available.

PHP Installation

PHP has been tested using a Google Nexus One and a Notion Ink Adam. The result of phpinfo() can be viewed here.

This is work in progress but it looks promising although not all PHP pages will be working but basic stuff should.

So stay tuned, I hope to finish this in a couple of weeks :)

Note:
PAW for Android 0.58 is now available at the Android Market.
The PHP plug-in is also available.

Update
If you have trouble downloading the PHP plugin from the original location, you can try the alternate link at Google Drive: PAW Plugins