Weavestream
Start Self-Hosting View on GitHub
5 min read

Custom API Integration: Pull Any JSON API Into Your IT Documentation

Weavestream's Custom API driver reads records from any JSON-over-HTTPS API into the asset layout you choose, with authentication, pagination, field mapping, dry runs, and scheduled syncs.

integrationscustom-apiapimspasset-managementit-documentation
01 / Post5 min read · 1,018 words

Every MSP has at least one system that never made it onto anyone’s integration roadmap. It might be a homegrown tool, a small RMM, an internal CMDB, or a mail server with a user list that only exists behind an API. The data is useful, but it lives somewhere your technicians don’t look when they open a client’s documentation.

Weavestream now has a Custom API integration driver for exactly this situation. If a source speaks JSON over HTTPS, you can describe it once and sync its records into the same asset layouts you already use.

What It Is (and Isn’t)

The name can be misleading, so it’s worth being precise. This is not an API that Weavestream exposes for other tools to call. It is a driver that lets Weavestream call your source API and read records from it. The source never receives a Weavestream key.

It is also read-only and pull-only. Weavestream makes the requests, and records flow one way into the asset layout you choose. Nothing is written back to the source.

Each integration covers one resource, called Items. Items can be devices, users, licenses, or whatever record type the source offers. Because each integration fills a single asset layout, a second record type means a second integration.

It Behaves Like Every Other Integration

A custom source doesn’t become a second-class citizen. You get the same features as the built-in providers:

  • Company mapping, so records land under the right client
  • Field mappings into an asset layout of your choice
  • Priority, for when several sources describe the same thing
  • Schedules and run history
  • A read-only section on the asset page

That means a record from your internal tool can sit next to the documentation, credentials, and procedures your team already keeps for the client.

Setting It Up

  1. In Admin → Integrations, create a new integration and choose the Custom API driver.
  2. Enter a display name, the base URL, and an authentication method.
  3. Define Organizations and Items, using each section’s Test button as you go.
  4. Map organizations to companies.
  5. On Items fields, choose the target asset layout, map fields, and set match keys.
  6. Run a Dry run, then Run sync now. Set the status to Active for scheduled syncs.

A new integration starts paused, so nothing runs until you say so. If you’d rather not start from scratch, you can begin from a JSON definition file with Import definition. The browser validates the file and reports problems before anything is saved. The documentation includes two example definitions: a cursor-paginated API with organizations from an endpoint, and a page-number API with a single organization.

Authentication That Matches Real APIs

Five methods are supported:

  • Bearer token
  • API key header, with a header name and value of your choosing
  • User name and password (Basic)
  • OAuth client credentials, with a token URL, optional scope, client ID and secret
  • No authentication, for sources that don’t need it

Credentials are write-only. Once saved, they can’t be read back, and rotating them means entering the values again. If you change the base URL or token URL to a different host, you must re-enter credentials in the same save, so saved secrets are never sent to a host you didn’t intend.

The base URL must use HTTPS, and requests can’t leave that host or path prefix. APIs on a private network need their address range added to EGRESS_ALLOWED_PRIVATE_CIDRS by whoever runs your instance. Plain HTTP is not supported, and neither are API keys in the query string.

Handling Multiple Clients

How do the records in a source map to your clients? Weavestream supports three modes:

  • From an endpoint: the API lists its organizations, and Weavestream runs each resource request once per organization.
  • From a field on each record: every record carries an organization ID, and Weavestream splits the results by company.
  • One organization: everything goes into a single “All records” organization.

Pick the mode that fits the source, then map each organization to a Weavestream company. If you change the mode after mapping, the organization IDs change too, so review your mappings after saving.

Real-World Pagination

APIs rarely return everything in one response, so the driver supports the common styles: none, page number, record offset, cursor, next URL, and Link header. Requests can use GET, or POST with a fixed JSON body, and placeholders such as {orgId}, {cursor}, {page}, {offset}, and {limit} are URL-encoded for you.

It is also deliberately cautious. A repeated page, cursor, or URL stops the run with an error, and so does a malformed cursor or URL. This is intentional: a broken response should never be able to make Weavestream think records have disappeared and mark assets as stale.

Mapping Fields

Fields are located with simple dotted paths, such as data.items or addresses[0].value. Each item needs an ID path and a name path. An optional match value path, such as a serial number or an email address, lets Weavestream link incoming items to assets you already have instead of creating duplicates.

You can map up to 64 fields, each with a key, label, path, and type: text, number, boolean, date, datetime, URL, email, or IP address. An optional “Open in” link template adds a link from each record back to the source application.

Safe to Experiment With

Tests show five sample records and the keys of the first one, with sensitive-looking values masked. Raw responses are never displayed or logged. Combined with the paused starting state and the dry run, you can get a definition right before a single asset is created or changed.

Check the limits section before designing a large sync: definitions are capped at 64 KiB, with up to 16 extra headers and 10,000 records per page.

Where to Start

Choose one small source you’ve been meaning to document properly: a server inventory, an internal user list, a license tracker. Import one of the example definitions, adjust the paths to match the source’s response, test each section, and run a dry run for a single client.

Read the full Custom API documentation for every setting, and see the other supported integrations if your source already has a built-in driver.

02 / Keep reading

Latest from the blog