Java OpenTelemetry Instrumentation

This document contains instructions on how to set up OpenTelemetry instrumentation in your Java applications. OpenTelemetry, also known as OTel for short, is an open source observability framework that can help you generate and collect telemetry data - traces, metrics, and logs from your Java application.

OpenTelemetry Java is the language-specific implementation of OpenTelemetry in Java.

Once the telemetry data is collected, you can configure an exporter to send the data to SigNoz.

There are three major steps to using OpenTelemetry:

• Instrumenting your Java application with OpenTelemetry
• Configuring exporter to send data to SigNoz
• Validating that configuration to ensure that data is being sent as expected.

There are two types of application instrumentation:

• Auto Instrumentation
  A completely automatic and out of box experience, with minimal code changes. For your Java application, we recommend getting started with auto instrumentation.

  ✅ Info

  If you are on K8s, you should checkout opentelemetry operators which enable auto instrumenting Java applications very easily.

• Manual Instrumentation
  It involves writing instrumentation using OpenTelemetry SDK and API manually. You would need to get a handle to an instance of the OpenTelemetry interface, acquire a tracer, and create spans manually. Manual isntrumentation might also be used along with auto instrumentation.

Let’s understand how to download, install, and run OpenTelemetry in Java. If you're using SigNoz cloud, refer to this section. If you're using self-hosted SigNoz refer to this section.

Requirements

Java 8 or higher

Send Traces to SigNoz Cloud

OpenTelemetry provides a handy Java JAR agent that can be attached to any Java 8+ application and dynamically injects bytecode to capture telemetry from a number of popular libraries and frameworks.

Based on your application environment, you can choose the setup below to send traces to SigNoz Cloud.

wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

OTEL_RESOURCE_ATTRIBUTES=service.name=<app_name> \
OTEL_EXPORTER_OTLP_HEADERS="signoz-ingestion-key=SIGNOZ_INGESTION_KEY" \
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.{region}.signoz.cloud:443 \
java -javaagent:$PWD/opentelemetry-javaagent.jar -jar <my-app>.jar

wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar

java -javaagent:$PWD/opentelemetry-javaagent.jar -jar <myapp>.jar

Send Traces to Self-Hosted SigNoz

You can use OpenTelemetry Java to send your traces directly to SigNoz. OpenTelemetry provides a handy Java JAR agent that can be attached to any Java 8+ application and dynamically injects bytecode to capture telemetry from a number of popular libraries and frameworks.

Steps to auto-instrument Java applications for traces

OpenTelemetry Java auto-instrumentation supports collecting telemetry data from a huge number of libraries and frameworks. You can check out the full list here.

1. Download the latest OpenTelemetry Java JAR agent
   Download the latest Java JAR agent. You can also use the terminal to get the file using the following command:

    wget https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
   

2. Enable the instrumentation agent and run your application
   If you run your Java application as a JAR file, run your application using the following command:

   OTEL_EXPORTER_OTLP_ENDPOINT="http://<IP of SigNoz Backend>:4317" OTEL_RESOURCE_ATTRIBUTES=service.name=<app_name> java -javaagent:/path/to/opentelemetry-javaagent.jar -jar  <myapp>.jar
   

   where <app_name> is the name you want to set for your application. path should be updated to the path of the downloaded Java JAR agent.

   In the above command, we are configuring the exporter to send data to SigNoz backend. By default, OpenTelemetry Java agent uses OTLP exporter configured to send data.
   

   Two things to note about the command:
   

   OTEL_EXPORTER_OTLP_ENDPOINT - This is the endpoint of the machine where SigNoz is installed.

   path/to - Update it to the path of your downloaded Java JAR agent.
   

   If you have installed SigNoz on your localhost and your Java JAR agent is saved at /Users/john/Downloads/, then the final command looks like:
   

   OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317" OTEL_RESOURCE_ATTRIBUTES=service.name=javaApp java -javaagent:/Users/john/Downloads/opentelemetry-javaagent.jar -jar target/*.jar
   

   Here’s a handy grid to figure out which address to use to send data to SigNoz.

   You can also specify environment variables in the following way:

   java -javaagent:/path/opentelemetry-javaagent.jar \
       -Dotel.exporter.otlp.endpoint=http://<IP of SigNoz Backend>:4317 \
       -Dotel.resource.attributes=service.name=<app_name> \
       -jar <myapp>.jar
   

In case you encounter an issue where all applications do not get listed in the services section then please refer to the troubleshooting section.

Validating instrumentation by checking for traces

With your application running, you can verify that you’ve instrumented your application with OpenTelemetry correctly by confirming that tracing data is being reported to SigNoz.

To do this, you need to ensure that your application generates some data. Applications will not produce traces unless they are being interacted with, and OpenTelemetry will often buffer data before sending. So you need to interact with your application and wait for some time to see your tracing data in SigNoz.

Validate your traces in SigNoz:

1. Trigger an action in your app that generates a web request. Hit the endpoint a number of times to generate some data. Then, wait for some time.
2. In SigNoz, open the Services tab. Hit the Refresh button on the top right corner, and your application should appear in the list of Applications. Ensure that you're checking data for the time range filter applied in the top right corner.
3. Go to the Traces tab, and apply relevant filters to see your application’s traces.

You might see other dummy applications if you’re using self-hosted SigNoz for the first time. You can remove it by following the docs here.

If you don't see your application reported in the list of services, try our troubleshooting guide.

Configuring the agent

The agent is highly configurable. You can check out all the configuration options available here.

Disabled instrumentations

Some instrumentations can produce too many spans and make traces very noisy. For this reason, the following instrumentations are disabled by default:

• jdbc-datasource which creates spans whenever the java.sql.DataSource#getConnection method is called.
• dropwizard-metrics, which might create very low-quality metrics data because of the lack of label/attribute support in the Dropwizard metrics API.

To enable them, add the otel.instrumentation.<name>.enabled system property: -Dotel.instrumentation.jdbc-datasource.enabled=true

Manual Instrumentation

For manual instrumentation of Java application, refer to the docs here.

Instrumentation using Otel buildpack (paketo) for Java

1. Clone OTel buildpack repo:
   

   git clone https://github.com/paketo-buildpacks/opentelemetry.git
   

2. Switch to config-binding branch:
   

   git checkout config-binding
   

3. Run the following command:
   

   scripts/build.sh
   

4. Now run below command to build a pack:
   

   pack build paketo-demo-app \
     --path /Users/makeavish/samples/java/maven \
     --buildpack paketo-buildpacks/java \
     --buildpack . \
     --builder paketobuildpacks/builder:base \
     --verbose --trust-builder \
     -e BP_JVM_VERSION=17 -e BP_OPENTELEMETRY_ENABLED=true
   

5. Pass environment variables to enable java agent OTEL_JAVAAGENT_ENABLED=true set exporter endpoint OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 and set service name OTEL_RESOURCE_ATTRIBUTES=service.name=javaApp

6. Other otel configurations can be updated by passing more environment variables. Refer to official docs for more such configurations.

7. Run docker command:

   docker run -d -p 8080:8080 -e PORT=8080 -e OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 -e OTEL_RESOURCE_ATTRIBUTES=service.name=javaApp -e OTEL_JAVAAGENT_ENABLED=true paketo-demo-app
   

Troubleshooting your installation

If spans are not being reported to SigNoz, try running in debug mode by setting OTEL_LOG_LEVEL=debug:

The debug log level will print out the configuration information. It will also emit every span to the console, which should look something like:

Span {
  attributes: {},
  links: [],
  events: [],
  status: { code: 0 },
  endTime: [ 1597810686, 885498645 ],
  _ended: true,
  _duration: [ 0, 43333 ],
  name: 'bar',
  spanContext: {
    traceId: 'eca3cc297720bd705e734f4941bca45a',
    spanId: '891016e5f8c134ad',
    traceFlags: 1,
    traceState: undefined
  },
  parentSpanId: 'cff3a2c6bfd4bbef',
  kind: 0,
  startTime: [ 1597810686, 885455312 ],
  resource: Resource { labels: [Object] },
  instrumentationLibrary: { name: 'example', version: '*' },
  _logger: ConsoleLogger {
    debug: [Function],
    info: [Function],
    warn: [Function],
    error: [Function]
  },
  _traceParams: {
    numberOfAttributesPerSpan: 32,
    numberOfLinksPerSpan: 32,
    numberOfEventsPerSpan: 128
  },
  _spanProcessor: MultiSpanProcessor { _spanProcessors: [Array] }
},

Sample Java Application

• We have included a sample Java application with README.md at Sample Java App Github Repo.

Frequently Asked Questions

1. How to find what to use in IP of SigNoz if I have installed SigNoz in Kubernetes cluster?

   Based on where you have installed your application and where you have installed SigNoz, you need to find the right value for this. Please use this grid to find the value you should use for IP of SigNoz

2. I am sending data from my application to SigNoz, but I don't see any events or graphs in the SigNoz dashboard. What should I do?

   This could be because of one of the following reasons:

   1. Your application is generating telemetry data, but not able to connect with SigNoz installation

      Please use this troubleshooting guide to find if your application is able to access SigNoz installation and send data to it.

   2. Your application is not actually generating telemetry data

      Please check if the application is generating telemetry data first. You can use Console Exporter to just print your telemetry data in console first. Join our Slack Community if you need help on how to export your telemetry data in console

   3. Your SigNoz installation is not running or behind a firewall

      Please double check if the pods in SigNoz installation are running fine. docker ps or kubectl get pods -n platform are your friends for this.

What Cloud Endpoint Should I Use?

The primary method for sending data to SigNoz Cloud is through OTLP exporters. You can either send the data directly from your application using the exporters available in SDKs/language agents or send the data to a collector agent, which batches/enriches telemetry and sends it to the Cloud.

My Collector Sends Data to SigNoz Cloud

Using gRPC Exporter

The endpoint should be ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu. Note that the exporter endpoint doesn't require a scheme for the gRPC exporter in the collector.

# Sample config with `us` region
exporters:
    otlp:
        endpoint: "ingest.us.signoz.cloud:443"
        tls:
            insecure: false
        headers:
            "signoz-ingestion-key": "<SIGNOZ_INGESTION_KEY>"

Using HTTP Exporter

The endpoint should be https://ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu. Note that the endpoint includes the scheme https for the HTTP exporter in the collector.

# Sample config with `us` region
exporters:
    otlphttp:
        endpoint: "https://ingest.us.signoz.cloud:443"
        tls:
            insecure: false
        headers:
            "signoz-ingestion-key": "<SIGNOZ_INGESTION_KEY>"

My Application Sends Data to SigNoz Cloud

The endpoint should be configured either with environment variables or in the SDK setup code.

Using Environment Variables

Using gRPC Exporter

Examples with us region

• OTEL_EXPORTER_OTLP_PROTOCOL=grpc OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.us.signoz.cloud:443 OTEL_EXPORTER_OTLP_HEADERS=signoz-ingestion-key=<SIGNOZ_INGESTION_KEY>

Using HTTP Exporter

• OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.us.signoz.cloud:443 OTEL_EXPORTER_OTLP_HEADERS=signoz-ingestion-key=<SIGNOZ_INGESTION_KEY>

Configuring Endpoint in Code

Please refer to the agent documentation.

Sending Data from a Third-Party Service

The endpoint configuration here depends on the export protocol supported by the third-party service. They may support either gRPC, HTTP, or both. Generally, you will need to adjust the host and port. The host address should be ingest.{region}.signoz.cloud:443, where {region} should be replaced with in, us, or eu, and port 443 should be used.