PirateBox Reloaded - Episode II

First of all thanks to all of you who gave feedback regarding the PirateBox for Android. The feedback was very useful and showed that there are problems when running the app on Android 4.x.
Meanwhile I've rooted my Galaxy Nexus (still on stock ROM) running 4.3, so I had the change for in depth testing. The result is version 0.4.0 of the app which includes a different handling of the dnsmasq process. I'll go a little into detail to describe the changes that have been made.

Those of  you not interested can jump directly to the end of this post where the download link is located. I've also added the questionnaire again. It would be really nice if you could provide feedback, it's  really useful.

Wrapping Dnsmasq

In previous version of the app the dnsmasq process was killed and restarted with the --address=/#/. This worked fine on Android 2.x devices but lead to a continuous restart of the AP (Access Point) on Android 4.x devices.
The new approach is different in the way that it does not kill the dnsmasq process but installs a wrapper script that is created before the AP stars and removed afterwards.
For this to work the app saves a copy of the file /system/bin/dnsmasq to /system/bin/dansmasq.pb.real. On start of the PirateBox service the wrapper script is created which overwrites the /system/bin/dnsmasq command. After the AP is started the original dnsmasq binary is restored.
The wrapper script looks like this:

#!/system/bin/sh
exec /system/bin/dnsmasq.pb.real --address=/#/192.168.43.1 $*

The wrapper calls the original dnsmasq binary which is now located at /system/bin/dnsmasq.pb.real with the --address and all other other parameters ($*) the system used to start dnsmasq.

If you look at the process list after starting the AP you should see that the backed up dnsmasq.pb.real is running instead of the real one (dnsmasq).
Here is the result of the ps | grep dnsmasq command:

shell@maguro:/ $ ps | grep dnsmasq
nobody    5088  121   928    496   ffffffff 00000000 S /system/bin/dnsmasq.pb.backup

New Preferences

The preference screen has two new additions. One is the IP address of the AP and the other is the option to restore the original dnsmasq binary.

New Preference Options

In previous versions the IP of the AP could determined after the AP was started and used to restart dnsmasq. This is no longer possible, because the wrapper script has to be created before the AP starts.

On Android most of the devices have the IP address 192.168.43.1 hard coded and I don't believe that there are a lot ROMs with different addresses. Should your AP start with a different IP address, you can still change this by using the new preference setting AP IP Number.

The second new preferences option is called Restore "dnsmasq" Binary. This will, as the name implies, try to restore the original dnsmasq binary. This should not be necessary to used, because the wrapper is removed and replaced by the original binary after the AP has been started. But in case thing do not work as desired when the device is used for standard tethering, this is the option to restore the binary.

Simple Widget

The new version also comes with a simple widget. It is not pretty ... but hey, I'm a developer not an artist ;)
Meanwhile I've updated the widget images in version 0.4.1, hope the widget is looking a bit better.

Below are the links to the app, the questionnaire and to the source code located at GitHub.

So I hope the new version is working better as the previous version and I'm looking forward for your feedback.

---

Source Code

The source code is now available on GitHub.

Download Links

Google Drive: PirateBox APKs

Questionnaire

The results can be viewed here, so you know on which configurations PirateBox is running.

A Look at Android Wear

Google released Android Wear a version of the Android OS specifically designed for wearable device like smart watches and the like.
Actual devices should follow shortly by the middle of this year starting with smart watches from Motorola and LG.

Along with the announcement of Android Wear Google released a preview version of the Android Wear SDK for developers to test and try the new OS.
Looking at the SDKs emulator and docs thing might look a little bit disappointing at first glance, if the expectation is that reals apps can be installed on an Android Wear device.
The Wear API is designed that the wearable device is showing notifications from an Android phone by using the since Android 4.3 Jelly Bean available Notification Access service. This service allows apps to receive information about notifications displayed on device notification bar. In earlier versions of Android this was only available by using the Accessibility feature of Android.

To connect a device to an Android Wear emulator, Google has provided an Android Wear Preview app for registered developers. Because using the above mentioned “Notification Access” service, this app only runs on devices with Android 4.3 or higher.
When the app is enabled all (non ongoing) notifications will be displayed on the Android Wear device with the ability to use rich notifications which allow to use action buttons for sending Intents back to the smart phone.
This design has the advantage that without any change current apps will work with Android Wear out of box.

The possibilities currently available with that notification approach are quite limited and can be viewed on the Android Wear developer page.

In summary developers can use the following notification types:

• Rich notifications including images and action buttons
• Grouped notifications
• Notifications with multiple pages
• Receive voice input from a notification

That seems to be it as far as the official API concerned. But the question arises (at least from a hacker’s perspective) … Can the Android Wear OS run regular apps?

If we have a look at the above linked developer page, it states:

“Caution: Do not install apps on the Android Wear emulator. The system does not support traditional Android apps and the result of running such apps is unpredictable.”

It is indeed possible to run apps with some exceptions. Network access is not possible and in the case of most apps will lead to a crash of the app. Web Views are also not supported. Apart from that, apps seem to work fine, but the small display size might be a problem.
Because this is a block for developers and hackers, let’s have a look how to run apps on the Android Device emulator.

First of all an APK of an app is needed that does not require network access. An example of such an app is the StopWatch & Timer app from Google Play.
The APK can be extracted from another Android device with a backup app. Using APK Backup of the PAW web interface is also an option.
The APK can be simply installed onto an emulator running Android Wear with the following command:

adb install

When the APK has been installed, the app StopWatch & Timer app can be started like this:

adb shell am start -n com.sportstracklive.stopwatch/com.sportstracklive.stopwatch.BigStopWatchActivity

After startup the app will show up on the emulator’s screen. Apart from the  small screen estate, the app seems to be working fine. Event the preference screen is working.

All in all Android Wear seems to be a nice addition to the Android family and summer seems to be getting interesting with new devices popping up.

Writing/Deleting Data with KitKat

Unfortunately Android 4.4 KitKat does restrict file access to the secondary SD card when it comes to writing and deleting files.

This restriction was actually introduced in Android 3.2 HoneyComb but was not enforced by device manufacturers.

With KitKat this seems to have changed (e.g. for Samsung devices) and so File-Managers including DavDrive can not write or delete files on the secondary SD card on some devices.

There is nothing that can be done from a software perspective, the only chance to change this is to root the device.

A detailed description can be found here:

External Blues: Google Has Brought Big Changes To SD Cards In KitKat, And Even Samsung Is Implementing Them

Thanks to Stefan for letting me know about this issue!

PirateBox Reloaded

It's been a while since the last PirateBox update. The last version was based on a PAW server plug-in  that didn't work well for all users.
Meanwhile I developed a stand alone PirateBox Android app that hopefully will work better.

The new app is based on CyanogenMod 7 and was tested on a Nexus One.
I have no other devices running CyanogenMod but I hope that the app will run too on different versions of that ROM.

CyanogenMod 7 and Nexus One

This post will describe the app in some detail. Technical details about the networking part will not be covered, because these were described in the earlier post. The techniques are the same, so there is no need to go through all of that again.
For those of you not interested, don't worry, here is the download link to the app straight away. Just download the APK and install it. A rooted device is necessary and it has only be tested with CyanogenMod 7.

So here is the Google Drive download link: PirateBox APKs

Running the App

For those still reading here is the more detailed part...
When you start the app you will  see the (not so spectacular) main screen which basically consists of an image and a start button. If your device is not rooted or the device is missing some prerequisites there will be a message telling so and you will not be able to proceed.

PirateBox - Switched off

After pressing the start button the app will start to setup the network. There is no need to start the hotspot beforehand, that will be done automatically. The app also take care to create an open hotspot and to name it appropriately.
After all is set up, you should see three icons underneath the start button and a notification in form of a scull.

PirateBox - Running

The icons show the active components that are configured during start up. These are, from left to right, the web server, the access point and the network configuration.

 Connecting via Browser

The PirateBox should show up as access point with the default SSID named  PirateBox - Share Freely. The name can be changed inside the setting, but we'll come to that in the next section.

PirateBox - Wi-Fi Access Point

After connecting to the access point you should be able to enter any HTTP URL inside your browser's address bar to get to the index page of the PirateBox.

PirateBox - Index page

App Settings

The app has a few settings. We will discuss them briefly.

PirateBox - Settings

Start On Boot: If you check that option, the PirateBox will start up after the device has been booted and the SD card is ready.

SSID Name: You can change the SSID name here. The app will try to change the name immediately. A restart should not be necessary. The default name is PirateBox - Share Freely as mentioned above.

Storage Directory: The directory which contains the uploaded files and the messages inside the shoutbox. You can locate the directory wherever you like. The directory /sdcard/pb_store is the default.

Enable Updates: On default the configuration and HTML files are updated with each new version of the app. If you don not want to do that, because you want to use your own configuration, un-check that option and no updates will be done.

iOS WISPr Support: iOS devices use WISPr request to detect if the device is indeed online. If not the device will not permanently connect to the hotspot. If that option is enabled the iOS device assumes it's online.

WP NCSI Support: Basically the same as for iOS devices. Windows Phone request a NCSI file to check if it's online.

App Broadcasts

This is a more advances topic and directed to developers which would like to interface with the app.
You do not need to read this section to use the app.
The app broadcast Intents to make indications  about the state of the PirateBox , of file uploads and shout messages.
Here are the broadcasts and the extra values provided.

Intent: de.fun2code.android.piratebox.broadcast.intent.server
Extras: state (boolean)
Description: Indicates the state of the web server.

Intent: de.fun2code.android.piratebox.broadcast.intent.ap
Extras: state (boolean)
Description: Indicates if the access point is running or not.

Intent: de.fun2code.android.piratebox.broadcast.intent.network
Extras: state (boolean)
Description: If the state is true, the networking (dnsmasq and iptables) are configured.

Intent: de.fun2code.android.piratebox.broadcast.intent.shout
Extras: name (String), text (String)
Description: If a message is entered into the shoutbox, the name and text will be included inside the extras of this Intent.

Intent: de.fun2code.android.piratebox.broadcast.intent.upload
Extras: file (String)
Description: Upon file upload the extra will contain the FQN of the file.

---

Current State

The app is in an very early state. I've tested as good as I could. There are surely bugs and  in some cases the app might not work at all.

New updates of the APK will show up in the Google Drive folder linked below.
So if you are interested, check the folder from time to time.

Feedback is always welcome!

Source Code

The source code is now available on GitHub.

In Case of Errors

If the PirateBox does not work as expected, please send me a mail describing your setup and include the LogCat output ... thanks

Mail address: jochen [at] fun2code.de

Download Links

Google Drive: PirateBox APKs

Questionnaire

If you tried the app, please provide feedback by filling out the following questionnaire ... thanks!
The results can be viewed here, so you know on which configurations PirateBox is running.

Chatting with Web Sockets

Web Sockets enable direct socket connections between a web browser and a web server.
Newer versions of PAW support text based Web Sockets. This post shows how to build a small Web Sockets based chat server.

If you have ever tried coding a web based chat server you have surely noticed that the task is not as easy as it might seem at the first look. You have to deal with sessions and have to synchronize clients and so forth. That’s one of the reasons why I never did that ;).
We will see that with Web Sockets there are only a few lines of code necessary on the server side.

I will not go into much detail when it comes to the HTML part, this is standard and everybody can google that.
What I will explain in detail is the PAW setup and the server side BeanShell script included in implementing the chat server.

Configuring the Web Socket Handler

To enable Web Socket support for your PAW server, we have to insert and configure the Web Socket handler inside the handler.xml file. The file is located inside the conf folder of your PAW installation directory . The installation directory is normally /sdcard/paw. Inside the handler.xml file the installation directory is referenced as [PAW_HOME].

The handler can be places right in front of the configuration file inside the <handlers>...</handlers>
tag:

<handler status="active">
    <name>WebSocket Handler</name>
    <description>WebSocket Handler</description>
    <removable>true</removable>
    <id>websocket</id>
    <files/>
    <params>
      <param name="websocket.class" value="org.paw.handler.android.websocket.WebSocketHandler" />
      <param name="websocket.basedir" value="[PAW_HOME]" />
      <param name="websocket.config" value="[PAW_HOME]/webconf/websocket.conf" />
    </params>
  </handler>

There are two important parameters in that handler definition… basedir and config.

Let’s start with config. The parameter config specifies the location of the Web Socket definition file. This file defines rules that describe which BeanShell script (or Java class) to call if a Web Socket connection requesting a specific protocol comes along.

A Web Socket communication can specify a protocol to use, which makes it possible for the server to react differently even if listening to the same port.
A common protocol type is chat which the client sends to the server within the header when the connection is established. You can also define your own protocol, it’s just a text.

In our example the web browser will send a different protocol for each chat room which makes it possible for the server to distinguish them. So if a chat room is called talk, the protocol will be chat.talk.

The other parameter basedir defines the base directory of the BeanShell scripts which are defined inside the Web Socket configuration file.

In the next step, we’ll create the Web Socket config file.

Web Socket Config File

As defined in the handler definition, create a file called [PAW_HOME]/webconf/websocket.conf with the following content:

# Format:
# Protocol:Bsh script (.bsh extension) or class name
#
chat.*:html/websocket/websocket_chat.bsh

That’s it, actually quite simple. I’ll quickly explain the syntax.
The parameters are devided by colons.

First parameter is the protocol which is defined as a regular expression. So in our case the server will call the script html/websocket/websocket_chat.bsh for all protocols starting with chat. Now you already know the second parameter which is the BeanShell script to call.

Actually you can also use a Java class (or performance reasons) but this will not be handled inside this post. More about how to get this working with Java classes can be found inside the PAW Functions section of the PAW web interface.

The next step is to create the BeanShell script.

The BeanShell Script

Before creating the script, some infos in advance.
The handler sets some variables before it calls the script. These variables are action, message, sockets and socket.
The action variable tells us something about the state of the communication. Possible values are connect, disconnect  or text. The state important for us is text because it indicated that a text message has been sent by a client.

If text is set, there is also a variable called message which contains the message itself.

Let’s now have a look at the code, because I think will make things clearer.
Create a file called [PAW_HOME]/html/websocket/websocket_chat.bsh
 with the following content:

import de.fun2code.android.pawserver.websocket.WebSocketMessage;

if(action.equals("text")) {
    for(sock : sockets) {
        try {
            WebSocketMessage.sendMessage(message, sock);
        }
        catch(e) {
          sock.close();
        }
    }
}

That’s the complete server side code. First thing that is checked is if the action variable is set to text which indicates that a message text is available inside the message variable.

If a message is present the code iterates over the sockets collection. The sockets collection contains all client socket connections that have the same protocol.
The socket collections are grouped by Web Socket protocol names, so we are sure that all sockets inside the collection belong to the same chat room.
As mentioned above, the web browser sends different Web Socket protocol names for each chat room (chat.>chatroom>).

So all that is left is to send the incoming message to all connected client. This is done by the following line:

 WebSocketMessage.sendMessage(message, sock);

The try/catch block is there to close a socket if something goes wrong.

One variable that is not present in the code and therefore was not handled until now is the socket variable. This variable contains the socket of the client that delivered the message.

The HTML File

The only thing that is missing is now the HTML file that is responsible for opening the Web Socket connection from the web browser. This is actually a XHTML file because it contains some additional BeanShell code.

The file can be downloaded from the link below. Just copy it into the [PAW_HOME]/html/websocket/ directory.
Now you can start your browser and call the XHTML file by inserting the following address into your browser’s address bar:

http://<ip>:<port>/websocket/chat.xhtml

Web Socket Chat

If everything goes well, the Web Socket chat server should be up and running.
In case of questions, write a comment or send me a mail.

Happy coding :)

Links

XHTML File: websocket_chat_xhtml.zip

PAW - Build Your Own Server

A lot of people have been asking about the PAW source code.
PAW is not Open Source, so (for the time being) you cannot get the source.

Nevertheless … it is now possible for developers to build their own PAW based apps for private and educational purposes. If you plan to use PAW in a commercial product, please drop me a line.

To make this easy, the PAW source code has been restructured.
A sample project along with documentation and a sample APK can be downloaded form the links listed below.
The process of creating a PAW based app should quite easy. Please read the linked documentation for detailed instructions.

 … happy coding! :)

Links:

Documentation: Build_Own_PAW_Server.pdf

Eclipse Project: BuildOwnPawServer.zip

Sample APK: BuildOwnPawServer.apk

PAW - Using other Languages with CGI

This post is about using other languages apart from BeanShell and PHP. PAW includes a CGI Handler to execute scripts by using the Common Gateway Protocol (CGI). This handler is also used by the PHP plugin to call the CGI version of PHP. The handler is defined inside the file PAW_HOME/conf/handler.XML. Instead of using PHP we will use Perl to create a simple web page.

Finding the Executable Path

First thing to do is to install the SL4A and Perl APK from the Android Scripting Project: http://code.google.com/p/android-scripting/downloads/list After installing the APKs and extracting Perl it's time to find out where the Perl executable is located. One way to do it is to execute the Perl interpreter and to use a terminal emulator and run the ps command. In the case of Perl the executable is located here: /data/data/com.googlecode.perlforandroid/files/perl/perl

CGI Configuration

Now it's time to edit the PAW handler configuration. Open the file PAW_HOME/conf/handler.XML and add the following lines below the Basic Auth Handler configuration:

 <handler status="active">
    <name>Perl CGI  Handler</name>
    <description>Perl CGI  Handler</description>
    <removable>true</removable>
    <id>perlCgiHandler</id>
    <files />
    <params>
      <param name="perlCgiHandler.class" value="org.paw.handler.AndroidCgiHandler" />
      <param name="perlCgiHandler.root" value="[PAW_HOME]/html" />
      <param name="perlCgiHandler.suffix" value=".pl" />
      <param name="perlCgiHandler.runwith" value="/data/data/com.googlecode.perlforandroid/files/perl/perl" />
      <param name="perlCgiHandler.sl4a" value="true" />
      <param name="perlCgiHandler.ENV_TMPDIR" value="[PAW_HOME]/tmp" />
    </params>
  </handler>

Let's have a look at the parameters. The class parameter defines the handler to use. In our case this is the CGI Handler. The next parameter is the root parameter which specifies the directory the scripts are located. Subdirectories are also included. The suffix parameter defines the filename extension for the scripts. The runwith parameter is the most important one, it defines the location of the CGI binary. If the sl4a is set to true the environment variables AP_HOST and AP_PORT are set. Because this information is read from logcat entries, SL4A should be started right before the startup of PAW. Otherwise it might not possible for the handler to extract the SL4A configuration. The last parameter sets the TMPDIR environment variable. All parameters starting with ENV_ are used to set environment variables. Now the PAW configuration is complete and we can start to write a test CGI script.

The CGI Script

We will create a test CGI script inside the PAW_HOME/html directory called test.pl. The test script looks like this:

print "Content-type: text/html\n\n";

print "Hello world!\n";

After starting PAW and entering the URL http://<ip-address>:8080/test.pl into the address bar of the browser the following text should be displayed:

Hello world!

Conclusion

The Android Scripting Project offers many more interpreters. It should be possible use many of them together with PAW, as long as you can get them to work from the command line.