Frequently Asked Questions

General Questions

What is Appose?

Appose is a library for interprocess cooperation with shared memory. It enables easy execution of code in different programming languages and environments without copying large data structures. The guiding principles are simplicity and efficiency.

What languages does Appose support?

Currently, Appose has implementations for:

Java - Call Python, Groovy, or other languages from Java
Python - Call Java/Groovy or other languages from Python

Each language implementation can invoke workers in other supported languages.

Why use Appose instead of other IPC solutions?

Appose is designed to be simpler than alternatives like Apache Arrow, Google Protocol Buffers, or NATS.io, while still supporting:

Zero-copy data sharing via shared memory for large tensors
Dynamic environment construction with conda, pixi, or uv
Minimal dependencies - one self-contained library per language
Simple JSON-based protocol for easy debugging and custom workers

What was Appose originally designed for?

Appose was written to enable easy execution of Python-based deep learning from Java without copying tensors. However, its utility extends to any scenario requiring interprocess communication with shared memory.

Installation and Setup

Where are environments stored by default?

By default, Appose environments are cached at:

~/.local/share/appose/<env-name>

This means environments only need to be built once and are reused across runs.

You can specify a custom directory when building:

Environment env = Appose.pixi()
    .conda("python>=3.10")
    .base("/custom/path/my-env")
    .build();

env = appose.pixi() \
    .conda("python>=3.10") \
    .base("/custom/path/my-env")
    .build()

Can I use an existing conda/pixi environment?

Yes! Use Appose.wrap() or appose.wrap():

import java.io.File;

Environment env = Appose.wrap(new File("/path/to/existing/env"));

env = appose.wrap("/path/to/existing/env")

Appose will auto-detect whether it’s a pixi, conda, or system environment.

Which environment builder should I use?

Pixi (recommended) - Modern, fast, supports both conda and PyPI packages
Mamba/Conda - Traditional conda environments, widely compatible
uv - Fast Python virtual environments, PyPI packages only
System - Use existing system installations, no environment management

For most new projects, Pixi is recommended due to its speed and flexibility.

Usage Questions

How do I pass data between processes?

Use the inputs and outputs maps:

task = groovy.task("x * 2")
task.inputs["x"] = 42
task.wait_for()
result = task.result()  # 84

Task task = python.task("x * 2", Map.of("x", 42));
task.waitFor();
Object result = task.result(); // 84

For large arrays/tensors, use shared memory to avoid copying (see worker implementation docs).

How do I handle long-running tasks?

Use listeners to monitor progress and support cancelation:

from appose.service import ResponseType

def listener(event):
    if event.response_type == ResponseType.UPDATE:
        print(f"Progress: {event.current}/{event.maximum}")
    elif event.response_type == ResponseType.COMPLETION:
        print("Done!")

task = python.task(long_script)
task.listen(listener)

# Later, if needed:
if not task.status.is_finished():
    task.cancel()

Task task = python.task(longScript);

task.listen(event -> {
    switch (event.responseType) {
        case UPDATE:
            System.out.println("Progress: " + event.current + "/" + event.maximum);
            break;
        case COMPLETION:
            System.out.println("Done!");
            break;
    }
});

// Later, if needed:
if (!task.status.isFinished()) {
    task.cancel();
}

Can I run multiple tasks in parallel?

Yes, create multiple tasks and they will run concurrently:

with env.python() as python:
    task1 = python.task("import time; time.sleep(1)")
    task2 = python.task("import time; time.sleep(1)")

    # Both run concurrently
    task1.wait_for()
    task2.wait_for()

try (Service python = env.python()) {
    Task task1 = python.task("import time; time.sleep(1)");
    Task task2 = python.task("import time; time.sleep(1)");

    // Both run concurrently
    task1.waitFor();
    task2.waitFor();
}

How do I handle errors in tasks?

Check the task status and error message:

task = python.task("1 / 0")  # Will fail
task.wait_for()

if task.status == TaskStatus.FAILED:
    print(f"Error: {task.error}", file=sys.stderr)
    # Handle the error

Task task = python.task("1 / 0"); // Will fail
task.waitFor();

if (task.status == TaskStatus.FAILED) {
    System.err.println("Error: " + task.error);
    // Handle the error
}

Advanced Questions

Can I create custom workers?

Yes! Any program that implements the Appose worker protocol can be used as a worker. See Worker Protocol for details.

To use a custom worker:

custom = env.service("my-worker", "arg1", "arg2")

Service custom = env.service("my-worker", "arg1", "arg2");

Can I use other transport layers besides pipes+JSON?

No. Appose intentionally uses a simple stdin/stdout pipe-based JSON protocol. This keeps the implementation simple and the dependencies minimal.

If you need HTTP, Protocol Buffers, Arrow, etc., consider using those solutions directly. Appose focuses on being simpler than those alternatives while supporting shared memory and dynamic environments.

Can Appose work across machines?

No. Appose is designed for local interprocess communication with shared memory. For distributed computing, consider:

Apache Arrow for cross-machine data sharing
NATS.io for distributed messaging
gRPC for remote procedure calls

These are listed in the Alternatives and Complements section.

Can I add custom data type converters?

Currently, Appose handles data serialization via JSON, which supports:

Numbers, strings, booleans
Arrays/lists and objects/dictionaries
null values

For complex types like tensors, use shared memory where the name is passed as a string.

Plugin-based type converters are not currently supported to keep Appose simple and self-contained. See the FAQ in the main README for more rationale.

Troubleshooting

My environment build is hanging

Try enabling debug output to see what’s happening:

env = appose.pixi() \
    .conda("python>=3.10") \
    .name("my-env")
    .log_debug()  # This logs build output
    .build()

Environment env = Appose.pixi()
    .conda("python>=3.10")
    .name("my-env")
    .logDebug()  // This logs build output to stderr
    .build();

If the build is truly stuck, check:

Network connectivity (for downloading packages)
Disk space (environments can be large)
Package conflicts in your dependency list

The worker process crashes immediately

Check that:

The worker executable exists and is on the PATH
Required dependencies are installed (e.g., appose Python package for Python workers)
The environment is properly built

Try running the worker manually to see errors:

# For Python worker
python -m appose.python_worker

# For Groovy worker (from Java)
java -cp "appose.jar:..." org.apposed.appose.GroovyWorker

My task outputs are empty

Make sure your script explicitly sets outputs:

# Wrong (output not captured)
answer = 5 + 6

# Right (explicit output)
task.outputs["answer"] = 5 + 6

# Also right (single expression)
5 + 6  # Implicitly becomes task.outputs["result"]

// Wrong (output not captured)
answer = 5 + 6

// Right (explicit output)
task.outputs["answer"] = 5 + 6

// Also right (single expression)
5 + 6  // Implicitly becomes task.outputs["result"]

Tasks are slow to start

The first task in a service may be slower as the worker process starts up. Subsequent tasks should be faster.

If all tasks are slow:

Check if the environment needs to be built (first run)
Ensure the worker script doesn’t have expensive imports at the top level
Consider keeping the service open for multiple tasks instead of creating a new service each time

How do I debug worker communication?

Enable verbose logging and check stderr output from the worker process. You can also manually test workers by sending JSON to their stdin and reading stdout.

See Worker Protocol for details on the communication format.

Community and Support

Where can I report bugs or request features?

Use the shared issue tracker for all Appose implementations:

https://github.com/apposed/appose/issues

Where is the source code?

Main repository: https://github.com/apposed/appose
Java implementation: https://github.com/apposed/appose-java
Python implementation: https://github.com/apposed/appose-python

How can I contribute?

Contributions are welcome! Please:

Check the issue tracker for open issues or feature requests
Fork the appropriate repository
Create a pull request with your changes
Ensure tests pass and add new tests for new features

Is Appose production-ready?

Appose is actively developed and used in production scenarios. However, some features (like comprehensive shared memory support) are still being enhanced.

Check the release notes and version numbers for each language implementation for stability information.

What’s the license?

Appose implementations are open source. Check each repository for specific license information.