Skip to main content

Opensearch - Error fetching events using Observability / Event Analytics - error 503

Context​

When using Observability plugin (from Opensearch dashboard) for the first time, you may face this type of issue:

img

Error​

If you are new, you probably have tried this type of PPL query:

source = YOUR_INDICE

Then, if you click on "search" button, you will have the error presented in the previous screenshot.

If you look at the backend logs, you will have something like:

Event analytics fetch error:  { Error: [status_exception] ObservabilityObject YOUR_INDICE not found
    at respond (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/usr/share/opensearch-dashboards/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/usr/share/opensearch-dashboards/node_modules/lodash/lodash.js:4991:19)
    at IncomingMessage.emit (events.js:203:15)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 404,
  displayName: 'NotFound',
  message:
   '[status_exception] ObservabilityObject YOUR_INDICE not found',
  path: '/_plugins/_observability/object',
  query: { objectId: 'YOUR_INDICE' },
  body:
   { error:
      { root_cause: [Array],
        type: 'status_exception',
        reason:
         'ObservabilityObject YOUR_INDICE not found' },
     status: 404 },
  statusCode: 404,
  response:
   '{"error":{"root_cause":[{"type":"status_exception","reason":"ObservabilityObject YOUR_INDICE not found"}],"type":"status_exception","reason":"ObservabilityObject YOUR_INDICE not found"},"status":404}',
  toString: [Function],
  toJSON: [Function] }
note

the error looks very strange because the indice exists in your cluster

Root Cause​

In fact, the PPL query is wrong.

Fix​

Please use a full PPL query like:

source = YOUR_INDICE | stats count()

Then, you will see data:

img