PyQGIS Engineering & Automation

Fetching JSON from an API Without Freezing QGIS

Call an HTTP API from a QGIS plugin without blocking the window: QgsNetworkAccessManager, holding the reply, distinguishing transport errors from HTTP status…

TL;DR: issue the request through QgsNetworkAccessManager.instance().get(), keep a reference to the reply, and read the payload in the finished signal handler — the call returns immediately, so the interface never blocks, and the handler runs on the main thread where it is safe to touch widgets. This page is part of the network requests and remote data in plugins guide.

Complete Runnable Code

python
"""A small non-blocking JSON client for a QGIS plugin."""
import json
from qgis.core import QgsNetworkAccessManager
from qgis.PyQt.QtCore import QUrl
from qgis.PyQt.QtNetwork import QNetworkReply, QNetworkRequest

USER_AGENT = b"MyQgisPlugin/1.0 (+https://example.org/myplugin)"
TIMEOUT_MS = 15000


class JsonClient:
    """Issues JSON requests and reports results through callbacks.

    Holds each reply until its signal has fired, so nothing is collected
    mid-flight, and aborts everything outstanding when the plugin unloads.
    """

    def __init__(self):
        self._inflight: set[QNetworkReply] = set()

    def get(self, url: str, on_ok, on_error) -> None:
        request = QNetworkRequest(QUrl(url))
        request.setRawHeader(b"User-Agent", USER_AGENT)
        request.setRawHeader(b"Accept", b"application/json")
        request.setAttribute(QNetworkRequest.FollowRedirectsAttribute, True)
        request.setTransferTimeout(TIMEOUT_MS)      # never hang indefinitely

        reply = QgsNetworkAccessManager.instance().get(request)
        self._inflight.add(reply)

        def finished():
            self._inflight.discard(reply)
            try:
                if reply.error() != QNetworkReply.NoError:
                    on_error(reply.errorString())
                    return
                status = reply.attribute(QNetworkRequest.HttpStatusCodeAttribute)
                if status is not None and status >= 400:
                    on_error("HTTP %s from %s" % (status, url))
                    return
                try:
                    on_ok(json.loads(bytes(reply.readAll()).decode("utf-8")))
                except (UnicodeDecodeError, json.JSONDecodeError) as exc:
                    on_error("unreadable response: %s" % exc)
            finally:
                reply.deleteLater()

        reply.finished.connect(finished)

    def abort_all(self) -> None:
        """Cancel every outstanding request — call from the plugin's unload()."""
        for reply in list(self._inflight):
            reply.abort()
        self._inflight.clear()
What happens on each thread during a fetch A sequence diagram showing the plugin issuing a request, the access manager returning a reply object immediately so the interface keeps running, and the finished signal delivering the payload back on the main thread. plugin access manager reply get(request) starts the transfer returns at once UI stays live finished readAll(), deleteLater()

Architecture Breakdown

get() returns immediately

The call hands the request to Qt’s transfer machinery and returns a QNetworkReply straight away. Nothing has been transferred yet; the object is a handle to a transfer that will complete later. That is the entire reason the interface stays responsive — control returns to the event loop, which keeps painting and handling clicks while the bytes arrive.

The consequence people trip over is that reading the reply immediately after get() returns nothing. There is no data yet, and there will not be until finished fires.

Keeping the reply alive

Qt owns the reply on the C++ side, but the Python wrapper is refcounted like any other object. If the only reference was a local variable in the method that issued the request, it can be collected before the transfer completes, and the callback never runs — intermittently, and more often on slower connections, which makes it a memorable bug to diagnose.

The _inflight set in the example is that reference. Discarding it inside the handler keeps the set from growing, and deleteLater() releases the C++ object once Qt is finished with it.

Classifying the failure

There are four distinct ways this call fails and they surface in different places.

What each failure looks like to your code A grid of four failure modes — a timeout, a DNS failure, an HTTP error status and malformed JSON — showing where each surfaces and what the user should be told. surfaces as tell the user timeout reply.error() the service is slow or unreachable DNS failure reply.error() check the endpoint or the proxy HTTP 4xx / 5xx a status attribute the request or the service is wrong malformed JSON a parse exception the service returned something unexpected

reply.error() covers transport problems — timeouts, DNS, refused connections. An HTTP error status is not a transport error: the request succeeded and the server said no, so error() may well be NoError while the status attribute is 404. Checking only one of the two is how a plugin ends up trying to parse an error page as JSON.

Choosing the Right API for the Call

QGIS offers two request APIs and a third option that avoids the question entirely, and picking between them is decided almost completely by which thread the call starts on.

Which request API this call needs A decision tree on where the call is made from: the asynchronous manager on the main thread, the blocking request inside a task, and neither when a provider can fetch the data for you. Where is this call being made from? the GUI thread QgsNetworkAccessManager signal on finish inside a task QgsBlockingNetworkRequest runs its own loop it's a map service let the provider WFS, WMS, OGC API

The asynchronous manager is right whenever the call originates on the GUI thread, which in a plugin means anything triggered by a button, a menu item or a canvas signal. Inside a QgsTask the calculus changes: the worker thread is allowed to block, and QgsBlockingNetworkRequest gives you a straight-line function that returns a payload, which is far easier to write correctly than a callback chain.

The third branch is worth checking before writing any of it. If the remote data is a WFS, WMS or OGC API endpoint, the QGIS provider already speaks that protocol, handles paging and caching, and gives you a layer rather than a dictionary.

Wiring It Into a Plugin

python
class MyPlugin:
    def __init__(self, iface):
        self.iface = iface
        self.client = JsonClient()

    def fetch_status(self) -> None:
        """Triggered from a toolbar action; returns before the response arrives."""
        self.iface.messageBar().pushInfo("My Plugin", "Fetching…")
        self.client.get(
            "https://example.org/api/status",
            on_ok=lambda data: self.iface.messageBar().pushSuccess(
                "My Plugin", "Service reports %s" % data.get("state", "unknown")),
            on_error=lambda message: self.iface.messageBar().pushWarning(
                "My Plugin", "Could not reach the service: %s" % message))

    def unload(self) -> None:
        self.client.abort_all()

Both callbacks touch the message bar, which is only safe because finished is delivered on the main thread. That guarantee is what makes the callback style workable in a GUI plugin at all.

Production Best Practices

  • Set a transfer timeout. Without one a stalled connection can hang for minutes.
  • Hold a reference to every reply until its signal has fired.
  • Check error() and the HTTP status. They are different failures.
  • Call deleteLater(), not del; the object belongs to Qt.
  • Abort outstanding requests in unload(), or a late callback will call into a dead plugin.
  • Never parse a response you have not status-checked. An error page is valid HTML and invalid JSON, and the parse error hides the real problem.

Frequently Asked Questions

Why does my callback never fire?

The two usual causes are a collected reply and a connection made after the signal had already been emitted. The first is fixed by holding the reference; the second happens when a cached response completes synchronously, before connect() runs — connect first, then keep the reference, as the example does.

If neither applies, check that the event loop is actually running. In a script executed from the Python console the loop is live; in a standalone script it may not be, and callbacks are delivered only when it spins.

How do I do a POST instead?

QgsNetworkAccessManager.instance().post(request, payload) takes the body as a QByteArray. Set the Content-Type header explicitly — application/json for a JSON body — because Qt will not infer it, and a service receiving JSON labelled as form data usually rejects it with a status your code then has to interpret.

Can I make several requests at once?

Yes, and the class above already supports it: each get() produces an independent reply, and the set holds all of them. What you should not do is fire an unbounded number — a loop issuing one request per feature will exhaust connections and irritate the service. Batch where the API allows it, and otherwise limit concurrency to a handful.

How do I show progress?

Connect to the reply’s downloadProgress(received, total) signal. Note that total is -1 when the server sends no content length, which is common for streamed or compressed responses — show an indeterminate indicator in that case rather than computing a percentage from a negative number.

Should I cache responses?

For anything that changes slowly, yes. QgsNetworkAccessManager honours HTTP cache headers through Qt’s disk cache, so a well-behaved service costs nothing on a repeat call. When the service sets no cache headers, a small dictionary keyed on the URL with an expiry you control avoids turning every canvas refresh into a request.

What about services that need authentication?

Attach a stored authentication configuration to the request before issuing it, as described in storing credentials with QgsAuthManager. The configuration id is the only thing your plugin needs to hold, and the manager applies whatever the credential actually is.