Tag Archives: raspbian

OpenJFX with RaspberryPi

JavaFX on the Raspberry Pi is a particularly nifty platform to use when you need a nice looking GUI on a regular monitor or a touch-screen. The platform used to ship along with JDK8 for ARM directly and was bundled (last I saw it still was) with out of the box Raspbian.

However, from update 33,  Oracle decided to drop the support for JavaFX on the ARM distribution of their JDK, and stopped shipping it within as well.

However the story doesn’t end here. This change just expedited my idea to give OpenJFX a try, after which I wished I did that way sooner.

My test bed was a RaspberryPi with PiTFT and a touchscreen adapted JavaFX application. Previously I had implemented a headless start of the JavaFX application with fbcp running in the background, and having all the parameters for the touchscreen set right in order to get a nice and correct projection of the FX framebuffer. With OpenJFX, this was no longer needed, cause it is directly supported.

In order to quickly get and use OpenJFX on Raspbian, follow these steps:
– Get and flash a fresh Raspbian image. Make sure you have java and javac present.
– Download a built OpenJFX package (OpenJFX 8u## stable for armv6hf). I used @chriswhocodes OpenJFX builds. There are others listed at the OpenJFX community builds page.
– Extract the contents of the zip file. Copy the /jre/lib folder contents somewhere in your project (I simply copied everything to the project root folder).
– (include the /ext/jfxrt.jar file in your classpath if you compile in an environment without JavaFX present in the JDK)

Finally, when executing your java application, pass the following arguments to the JVM:
-Djava.ext.dirs=dir/to/jfx/ext (I used .dirs=ext because I unpacked the lib contents in the project root)
-Dglass.platform=Monocle
-Dmonocle.screen.fb=/dev/fb1  (only if you use a touchscreen (like the PiTFT)
-Dprism.order=sw (again only for touchscreen, but I’m not really sure. If you experience problems with eventual hardware rendering, use this)

The outcome was pretty pleasant. The UI was looking good and it adapted just fine, although be careful with the dimensions and the general conditions (see tips here).  Also, I had no need to calibrate the screen, it was working correctly from the first run. And last but not least, having not to use fbcp in the background is a huge performance boost and resource saver.

Examples:

Main screen with buttons LED light screen with color picker

 

DoorNFX: Touchscreen JavaFX 8 on Raspberry Pi

As of March 2014, Java8 is finally out there. Bunch of new features and improvements, not that they weren’t known previously, but good that they went official. The ones that I’m targeting with this blogpost are JavaFX, JDK8 on ARM devices, and their joint functionality.

The new JDK for ARM is targeted specifically for v6/v7 ARM HardFloat ABI devices running on Linux. The best and world-wide accepted example for this is the Raspberry Pi running on an OS like Raspbian. This JDK was around for some time with the early access program, so I had the chance to play around with it previously. However, for the example below, I’m using the official version.

JavaFX is, as the definition says, a set of graphics and media packages that enables developers to design, create, test, debug and deploy rich client applications that operate consistently across diverse platforms. In short, it’s a Java framework building Rich Internet or Desktop applications. Some of it’s features include:
– Pure Java API integrated in JavaSE: as from Java8, JavaFX is an integral part of the JRE and JDK. It’s API i in pure Java so it can be used any language that runs on the JVM.
– UI can be defined either programmatically or declaratively via FXML
– Interoperable with the old Swing
– All UI components can be styled with CSS
– New theme ‘Modena’ which makes the UI look very nice fora change
– General 3D features, hardware acceleration support
– WebView component which allows two-way interfacing (Java to JavaScript and vice-versa)
– Canvas and printing API, support for RichText
The easiest way to explore JavaFX is to play around with the Ensamble app on the Oracle web site.

THE DEVICE

DoorNFC GUI

So, as an example, I decided to make my NFC PN532 Java port to some usage and make some exact device out of it. My idea was to make a protected door access node which reads NFC Tags, prompts for a user code, authenticates it against some remote server and grants or declines access based on the output.

The core of the device is a RaspberryPi model B. The GPIO section has more than enough options for connecting multiple external devices. For the device, I’m using two such devices which are made specifically for the RaspberryPi: PiTFT and ITEAD PN532 NFC module.

HARDWARE

DoorNFC - Device

The touchscreen used is the adafriut 2.8” PiTFT resistive touchscreen with 320×240 resolution. It is a actually a Pi HAT device with a socket the same as the raspberry pi. Its assembly is very easy, and it’s usage with the Raspbian OS is relatively simple. For communicating with the RaspberryPi it uses the SPI interface.

The ITEAD NFC Module is a PN532 based board with an integrated antenna. It exposes the same functionality as all the other PN532 boards and uses either SPI, I2C or UART for communication. However, this device also has a native RaspberryPi header interface. One bad thing is, this interface can only work with SPI. Since the SPI and the same channel is already taken by the PiTFT, I made some alteration of the NFC module in order to patch it to work with the same header but by using I2C. I’ve described that procedure in my previous blogpost.

OPERATING SYSTEM

As the core for starting the device, I used the pre-built adafruit image of the Raspbian OS. This image is described in details in the adafruit tutorials section. Basically, it is a Raspbian OS with the patched kernel, driver and necessary configuration to enable and use the PiTFT. Besides all that, it also comes with JDK8 and nicely split GPU/CPU memory which is the core need for running JavaFX applications on the Pi.

With only the image however, the job for configuring the device is not done. First, the FrameBufferCopy tool (fbcp) will be needed:

Then,  the start-up console needs to be disabled. Do this by removing the fbcon map and fbcon font settings in /boot/cmdline.txt.

Next, and this is the trickiest part, the display needs to be adaptad to be with the same format as the PiTFT. The touchscreen is designed to be in portrait mode with resolution of 240×320. The original configuration of the X server done here is by rotating the display and re-calibrating the touchscreen. JavaFX runs in a framebuffer and it’s not connected to X whatsoever. Therefore, the display and the touchscreen behavior work differently and wrong. This can be fixed by force-adjusting the display resolution to 240×320 and not rotating the screen by default. In order to do so, alter the settings in /boot/config.txt:

and by resetting the rotation in /etc/modprobe.d/: rotate=0

At the end, in order to enable I2C, modify /etc/modules by adding:

and comment out i2c in /etc/modprobe.b/raspi-blacklist.conf.

SOFTWARE

The software for the device is already on github: https://github.com/hsilomedus/door-nfx

There are two packages present:

Writing JavaFX code for the Pi is rather straight forward. The  most important aspects have to be met at start, as they are more environment related. Others are just tips. Some that I can mention:

  • The CPU/GPU memory split needs to configured correctly in order to achieve nicer performances (or even to get the JavaFX app up and running). 128MB for the GPU is a decent amount.
  • The JavaFX app will run in a framebuffer. This is maybe the biggest difference that you must have in mind. Running JavaFX apps on the Pi is not conditioned by the presence of an X server: they don’t run in a widget or a frame and can be invoked straight from a console. Even better, running a JavaFX app from an X session will most likely break it and freeze the UI after you exit it. Always execute the JavaFX app from a console, local or remote.
  • Because the app will run in a Framebuffer, make sure that you use and manage all the visual space that you have in your disposal and run it in full screen. You can still run it with fixed size, but then it will most probably end up centered on the screen.
  • JavaFX will register it’s own Keyboard and mouse handler and render a mouse pointer. If you have some settings done in X that change the behavior of the mouse or the keyboard, they will not be present here. E.g.: a major problem with the touchscreen was the initial rotation. The screen was rotated, but the touchscreen was only calibrated for that in X. That’s why the settings here are reversed and the display is in portrait.
  • Last but not least: JavaFX runs in its own thread. If you are to populate other heavier operations from the main routine or an event handler, do it in a different thread. If you need to alter something UI related from a different thread, use Platform.runLater.

You can see some examples already implemented in the source code. It’s not very pragmatic or anything, but enough to get the idea and to get the device working.

A general frame of a basic JavaFX app looks something like this:

The other part of the code is the pi4j usage. Here I’m using the managed way to access the hardware aspect of the Pi and send/receive data through it:

  • I2C is used for communicating with the NFC PN532 module. The API is simple but the hard part is maintaining the protocol set by the device manufacturer:

    The adress of the device can be given by the manufacturer of the device,  or you can look it up with the tool i2cdetect or something similar. See some tips in this adafruit tutorial.
  • General I/O pin provisioning can be also combined, regardless that both SPI and I2C are used. Just make sure that you don’t provision a PIN somehow that will break the other two interfaces. Again, here I’m using the managed API of pi4j:

Some remarks about using pi4j:

  • Since pins need to be provisioned (exported), the java process MUST be started with sudo, or else it will fail.
  • I2C is used for communication, so make sure that the device is enabled and not blacklisted
  • The communication is not reliable. Your app should be prepared for that and easily recover from misscomunications.

RUNNING AND CONCLUSION

For ease of access, I’ve added two shell scripts (build.sh and run.sh) to make my compile&test experience on the Pi bearable. The pi4j library is automatically added in the classpath in both compile and test, the java process in run with sudo and fbcp is run in parallel.

The performance of the app itself is so-so. I can’t really deduct a conclusion since fbcp is an important parameter, and it may alter the visual response. Overall it is usable, but still not on that level that I want to see.

Always bear in mind that the device is quite limited with resources, and the platform itself is still catching up. It would be great if some ideas done in OpenJFX like setting a target framebuffer or altering the touchscreen input are implemented in the Oracle JDK too. That way, the output will be independent and I would presume more efficient.

P.S. I’ve also done a different JavaFX app which reads RFID tags, runs on an HDMI monitor, and is used as a poll. The output is quite bigger, the solution is simpler, but the overall user experience is still similar.