Photo Editor on Java

Junzo SATO, University of Kumamoto, Japan
Chikara MIYAJI, Japan Institute of Sports Sciences, Japan
Mariusz JANKOWSKI, University of Southern Maine, USA

Abstract

  MathLink is an inter-application communication protocol that is used to link the Notebook Frontend with the Mathematica Kernel, and potentially, any external program. The authors used this software technology to link an interactive graphical user interface (GUI) with Mathematica to design a photo editing program, the Photo Editor. The Mathematica package "Digital Image Processing" was used to provide image manipulation and processing functionality. The advantages are that by replacing the Notebook Frontend, the mouse, menus and graphical tools are available in a manner similar to other popular applications of this type, and that user actions can be defined and extended within Mathematica. Recently, the release of J/Link has made it possible to implement a MathLink program in the form of an external Java program. The entire Photo Editor GUI was rewritten to take advantage of the machine-independence of Java.

Java Photo Editor

  The goal of writing the Photo Editor is to provide the supplemental photo editing environment for the Notebook Frontend. The most basic elements of the development are as follows:

1) Graphical User Interface (GUI):
  GUI parts, such as windows, menus, and buttons etc. are required to show image data and to manipulate it.

2) Interactivity:
  To introduce interactive operations between image data shown by GUI and Mathematica, handlers for user tasks supported by the GUI should be implemented.

3) Flexibility:
  The GUI components and user task handlers are written using the Mathematica language and are therefore extensible. Users benefit by not having to write source code in external programming languages.

  Before the introduction of J/Link, the authors used to write MathLink programs using C/C++ [Ref 1,2]. J/Link however not only eliminates most of tedious steps of the development, but also provides an easy-to-use event handling mechanism. The authors took the advantage of J/Link to implement the new Photo Editor. The "Installable Java" feature of J/Link was particularly useful since it allows for most of the code to be written as Mathematica packages rather than as Java classes [Ref 3]. The Java Photo Editor is based on three technologies: Java Swing, Serializer, and Digital Image Processing. Each of these is described below.

Java Swing Interface

  The Java Photo Editor itself consists of loaded Java objects. Plentiful machine-independent GUI parts are prepared in the Java Foundation Class Swing package. These classes are loaded through Mathematica. The Fig 1. shows the example.

Fig 1. Java Swing by "Installable Java"

Interaction with Serializer

  When the user handles GUI parts shown on the desktop, the interaction between the Java Photo Editor and Mathematica such as the event handler is treated by the J/Link application.

Fig 2. User Action

  For example, the normal way to implement the menu command handler in the form of a Mathematica expression is to call setHandler[] method of the MathActionListener. A menu item selection operation generates an action event in Java. Then the Java menu item object receives actionPerformed() message. Within this method, a pre-installed Mathematica function is invoked using a KernelLink object. The image of the menu usually disappears immediately after the user click.

Fig 3. J/Link Environment

  However, if the Java environment doesn't get chance to hide the menu while the MathKernel is evaluating the installed handler, it remains on the desktop. This undesirable situation occurs when the user implements some time-consuming image processing command.

  In principle it should be sufficient for the menu command handler to inform the MathKernel which command was selected. At this point the menu handler should release the GUI object. The time-consuming part of the calculation should be done after the menu handler has finished its task. To make it possible to implement this type of behavior, the serializer.exe is adopted in the Java Photo Editor.

Fig 4. Serializer Environment

  When the installed menu handler is executed in the MathKernel through the Java Photo Editor, i.e., J/Link, the expression for letting serializer.exe send a notification to the MathKernel is evaluated. The handler is freed and the selected menu item is hidden. The serializer.exe creates a notification objTrigger[command, 7] and relays this expression to the MathKernel. Inside the objTrigger[command, 7] implemented in the Mathematica Notebook, image processing function is called.

This is the definition of the menu command handler of "Threshold".

List 1. Menu Command Handler

  The Java object that handles the links in the serializer environment is called slink. The Java message doCommand[] is the function to generate a notification objTrigger[command, 7]. The kThreshold is set to 7. Here is the Java source code of the doCommand() method. The field "sym" is a String having the value "objTrigger".

public int doCommand(int num)
{
    if (ml!=null) {
        try {
            ml.putFunction("EvaluatePacket",1);
            ml.putFunction(sym,2);
            ml.putSymbol("command");
            ml.put(num);
            ml.endPacket();
            ml.flush();
            return 0;
        } catch (MathLinkException e) {
            System.out.println(e.getMessage());
            return 1;
        }
    } else {
        return 1;
    }
}

List 2. doCommand() method in Java

  Figure 5 shows the output of the packet monitor by serializer.exe. EvaluatePacket[objTrigger[command, 7]] is shown.

Fig 5. Packet Monitor

  When the MathKernel receives the notification, the command is dispatched and DoEffect[] is evaluated. This function retrieves the marquee bounds from the target canvas, transports image data to the MathKernel, applies Threshold[] to the target ImageData, and shows the result image on the canvas.

List 3. Notification

  In fact, the adoption of the serializer environment has another benefit which isn't shown in the above example. Because the installed handler by setHandler[] is called at the timing of the Java environment, it would be possible for a flood of packets to be sent to the MathKernel. If the MathKernel couldn't handle them at an appropriate rate, the sufficient response might not be achieved. Using the serializer environment, we have a chance to filter events before sending them from the Java object to the MathKernel [Ref 4].

Digital Image Processing

  Digital Image Processing is a Mathematica add-on package. It covers a vast area of scientific and engineering knowledge. In our example, when objTrigger[command, 7] is evaluated, it invokes DoEffect[kThreshold], where one of functions Threshold[] in this package is used to apply some effect on the image. The function Threshold[] implements the following operator:

where is a point in a two-dimensional image matrix and is a user selected threshold value.

Fig 6. Result of threshold operation.

It is important to note that the Mathematica function DoEffect[] can be redefined. Flexibility exists all the time while the Java Photo Editor is in use.

Summary

  A photo editing program that supplements the Notebook Frontend has been created using J/Link. In this program three technologies, Java Swing, Serializer, and Digital Image Processing were combined in an appropriate way. Interactivity between the Java Photo Editor and the Mathematica was implemented without loosing the flexibility of the Mathematica's expression scriptability.

References

[1] J.Sato, C.Miyaji, and M.Jankowski: Interactive Photo Editor FrontEnd using
MathLink, IMS'99, 1999

[2] Chikara Miyaji: Interactive Graphics using MathLink, IMS'99, 1999

[3] Todd Gayley: J/Link User Guide, Wolfram Research, 2000

[4] C.Miyaji and P.Abott: MathLink--Network Programming with Mathematica,
Cambridge University Press, 2001