Memory Management and Garbage Collection for GIS Objects in PyQGIS

Stable PyQGIS plugins and batch automation scripts depend on understanding how Python’s reference-counting garbage collector and QGIS’s C++ parent-child ownership model interact — a topic central to the PyQGIS Core Architecture & Data Handling guide. Unlike pure-Python programs, QGIS operates on a hybrid runtime: Python binds to a heavily optimised C++ core through SIP, and objects can be owned by either side depending on which API accepted them. When this boundary is mismanaged, the symptoms range from gradual resident-set growth in overnight batch jobs to immediate RuntimeError: wrapped C/C++ object of type ... has been deleted crashes and, in worst cases, silent segmentation faults. This page provides a structured workflow, production-ready code patterns, and diagnostic strategies so you can achieve predictable, stable memory behaviour in any long-running PyQGIS context.

Prerequisites #

Before applying the patterns below, confirm the following are in place:

• QGIS 3.16 or later — SIP ownership-transfer semantics and sip.isdeleted() are stable from this release onwards
• Python 3.9+ — tracemalloc and weakref.finalize are mature; type hints in snippets require 3.9+
• Core API familiarity — QgsVectorLayer, QgsRasterLayer, QgsFeatureIterator, QgsGeometry, QgsProject
• SIP installed — available in every QGIS distribution; verify with import sip; print(sip.SIP_VERSION_STR)
• A reproducible test dataset — a modest GeoPackage (10 k–100 k features) to stress-test cleanup loops without exhausting RAM before you reach the diagnostic section

Architecture and Internals: the SIP Ownership Boundary #

Every QGIS object that descends from QObject participates in Qt’s parent-child ownership tree. When a C++ object is given a parent, the parent deletes it on destruction — regardless of whether Python still holds a reference to the SIP wrapper. SIP tracks this with an internal ownership flag per wrapper instance: Python (the Python interpreter holds the only reference and will call the C++ destructor when the refcount drops to zero) or C++ (the C++ runtime owns the object; the Python wrapper is a non-owning proxy).

The diagram below shows the four lifecycle transitions that matter for GIS work.

The critical rule: once addMapLayer() or setParent() transfers an object to C++, Python’s del no longer destroys the underlying object — it only destroys the SIP wrapper. Conversely, if Python holds the last reference to an object the C++ side has deleted, any attribute access on that wrapper raises RuntimeError.

Step-by-Step Implementation #

1. Declare Ownership Intent at Instantiation #

Decide before writing any other code whether the object will be ephemeral (Python manages the lifetime) or persistent (QGIS project manages it). This single decision shapes every subsequent API call.

from __future__ import annotations

from qgis.core import QgsProject, QgsVectorLayer


def load_persistent_layer(path: str, name: str) -> QgsVectorLayer:
    """
    Load a layer and transfer ownership to QgsProject.

    After this call the Python variable is a non-owning proxy.
    Never call del on it — use QgsProject.removeMapLayer() instead.

    Returns:
        The registered layer (proxy); None if the layer is invalid.
    """
    layer = QgsVectorLayer(path, name, "ogr")
    if not layer.isValid():
        return None
    # Ownership transfers to QgsProject here.
    # addMapLayer returns the same object; re-assign for clarity.
    return QgsProject.instance().addMapLayer(layer)


def make_scratch_layer(name: str = "scratch") -> QgsVectorLayer:
    """
    Create a temporary memory layer that Python owns entirely.

    The layer is freed when this function's caller drops all references.
    Do NOT call addMapLayer() on it unless you intend to transfer ownership.
    """
    return QgsVectorLayer("Point?crs=EPSG:4326", name, "memory")

Working with QgsProject and the layer registry covers the full addMapLayer / removeMapLayer API contract, including the addToLegend flag that controls UI registration separately from C++ ownership.

2. Scope Iterators and Temporary Geometries #

QgsFeatureIterator holds a native provider cursor and an attribute value cache. Leaving the iterator open across unrelated logic, or retaining QgsGeometry references beyond the loop body, forces the provider to keep buffers alive. Always release both explicitly.

from __future__ import annotations

import gc

from qgis.core import QgsFeature, QgsFeatureRequest, QgsVectorLayer


def compute_total_area(layer: QgsVectorLayer) -> float:
    """
    Sum the geodesic area of every polygon feature, releasing geometry
    references immediately to cap peak native memory.

    Args:
        layer: A polygon QgsVectorLayer with a valid data provider.

    Returns:
        Total area in the layer's native CRS units squared.
    """
    request = QgsFeatureRequest()
    request.setNoAttributes()  # skip attribute cache entirely

    total: float = 0.0
    features = layer.getFeatures(request)
    try:
        feature = QgsFeature()
        while features.nextFeature(feature):
            geom = feature.geometry()
            total += geom.area()
            del geom  # release native coordinate buffer immediately
    finally:
        del features  # closes provider cursor and frees read-ahead buffers
        gc.collect()  # sweep any Python-level cycles created inside the loop

    return total

The nextFeature() pattern avoids constructing new QgsFeature wrappers each iteration, which matters when processing millions of features; see optimising feature iteration with getFeatures for the full performance analysis.

3. Orchestrate Cleanup in Long-Running Batch Scripts #

Python’s cyclic garbage collector defers collection until a threshold triggers. In tight processing loops that never yield to the event loop, the threshold may not trigger for thousands of objects. Couple periodic gc.collect() calls with explicit recreation of heavyweight QGIS context objects.

from __future__ import annotations

import gc
from pathlib import Path
from typing import Iterable

from qgis.core import (
    QgsCoordinateReferenceSystem,
    QgsCoordinateTransform,
    QgsCoordinateTransformContext,
    QgsProject,
    QgsVectorLayer,
)


def reproject_batch(
    paths: Iterable[Path],
    target_epsg: int,
    cleanup_interval: int = 50,
) -> None:
    """
    Reproject a collection of vector layers without accumulating PROJ
    pipeline objects or provider cursors across iterations.

    Args:
        paths:            Iterable of source file paths.
        target_epsg:      EPSG code for the output CRS.
        cleanup_interval: Force GC and reset transform context every N files.
    """
    target_crs = QgsCoordinateReferenceSystem.fromEpsgId(target_epsg)

    # Recreate a fresh transform context rather than reusing one across
    # hundreds of files — cached PROJ pipelines accumulate native heap.
    # See /pyqgis-core-architecture-data-handling/coordinate-transformations-and-crs-handling/
    ctx = QgsCoordinateTransformContext()

    for i, path in enumerate(paths):
        layer = QgsVectorLayer(str(path), path.stem, "ogr")
        if not layer.isValid():
            continue

        transform = QgsCoordinateTransform(layer.crs(), target_crs, ctx)
        # … reprojection logic …

        del transform
        del layer

        if i > 0 and i % cleanup_interval == 0:
            del ctx
            ctx = QgsCoordinateTransformContext()
            gc.collect()

    del ctx
    gc.collect()

Coordinate transformations and CRS handling documents the full QgsCoordinateTransform lifecycle including thread-safety constraints on the shared transform context.

4. Guard Cross-Boundary Dereferencing with sip.isdeleted() #

Any code path that stores a reference to a QGIS object and later acts on it asynchronously — in a signal handler, a QgsTask callback, or a timer slot — must check whether the C++ side has deleted the object before the callback fires.

from __future__ import annotations

import sip
from qgis.core import QgsMapLayer, QgsProject


def safe_layer_name(layer: QgsMapLayer) -> str | None:
    """
    Return the layer name only if the underlying C++ object is still alive.

    Signal handlers and QgsTask callbacks must call this guard pattern
    because QgsProject may remove layers between the signal emission and
    the Python slot execution.

    Args:
        layer: A QgsMapLayer proxy whose C++ lifetime is externally managed.

    Returns:
        The layer name, or None if the C++ object has been deleted.
    """
    if sip.isdeleted(layer):
        return None
    return layer.name()


# Example: safe signal handler
def _on_layers_will_be_removed(layer_ids: list[str]) -> None:
    for lid in layer_ids:
        lyr = QgsProject.instance().mapLayer(lid)
        if lyr is None or sip.isdeleted(lyr):
            continue
        print(f"About to remove: {lyr.name()}")

Advanced Patterns #

Weak References for Signal Callbacks #

Connecting a Python method or lambda to a QGIS signal creates a strong reference that keeps the originating object alive. Replacing the strong reference with a weakref.ref breaks the cycle.

from __future__ import annotations

import weakref
from typing import Callable

from qgis.core import QgsMapLayer, QgsProject


class LayerMonitor:
    """
    Monitors layer additions without preventing garbage collection of self.

    Stores only a weak reference to itself in the signal connection so that
    the monitor can be collected when the caller drops all strong references.
    """

    def __init__(self) -> None:
        self._weak_self: weakref.ref[LayerMonitor] = weakref.ref(self)
        QgsProject.instance().layerWasAdded.connect(self._make_slot())

    def _make_slot(self) -> Callable[[QgsMapLayer], None]:
        weak = self._weak_self

        def slot(layer: QgsMapLayer) -> None:
            monitor = weak()
            if monitor is None:
                # Self was collected; disconnect gracefully
                QgsProject.instance().layerWasAdded.disconnect(slot)
                return
            monitor._handle_layer(layer)

        return slot

    def _handle_layer(self, layer: QgsMapLayer) -> None:
        print(f"Layer added: {layer.name()}")

Signal and slot event handling in QGIS covers the full dispatch mechanism and the edge cases around disconnecting slots during signal emission.

Raster Block Management #

Raster processing is the dominant source of hidden native-memory leaks. Each QgsRasterBlock holds a large C++ heap buffer that is not tracked by Python’s memory profilers. Read blocks in the smallest tiles your algorithm allows, and delete them explicitly before moving to the next tile.

from __future__ import annotations

from qgis.core import QgsRasterDataProvider, QgsRectangle


def scan_raster_blocks(
    provider: QgsRasterDataProvider,
    band: int,
    tile_size: int = 256,
) -> None:
    """
    Iterate raster tiles without accumulating QgsRasterBlock objects.

    Args:
        provider:  An open QgsRasterDataProvider.
        band:      1-based band index to read.
        tile_size: Width and height of each tile in pixels.
    """
    extent: QgsRectangle = provider.extent()
    cols: int = provider.xSize()
    rows: int = provider.ySize()

    for row_start in range(0, rows, tile_size):
        row_end = min(row_start + tile_size, rows)
        for col_start in range(0, cols, tile_size):
            col_end = min(col_start + tile_size, cols)

            block = provider.block(
                band,
                extent,   # sub-extent would be computed from pixel coords in production
                col_end - col_start,
                row_end - row_start,
            )
            try:
                # … process block.data() …
                pass
            finally:
                del block  # releases the native heap buffer immediately

    # See /pyqgis-core-architecture-data-handling/memory-management-and-garbage-collection-for-gis-objects/preventing-memory-leaks-when-processing-large-geotiff-rasters/
    # for a complete worked example with sub-extent tiling on large GeoTIFFs.

weakref.finalize for Deterministic Teardown #

When the exact point of garbage collection matters — for example, to log a warning if a layer is freed without being removed from the project — weakref.finalize provides a deterministic teardown hook that fires when the Python object is collected, even if the cause is a reference cycle resolved by gc.collect().

from __future__ import annotations

import weakref

from qgis.core import QgsVectorLayer


def register_teardown_hook(layer: QgsVectorLayer) -> None:
    """
    Register a finalizer that logs if a layer is collected while still
    carrying a non-empty feature cache.

    The finalizer holds no strong reference to the layer, so it does not
    prevent collection.
    """
    layer_id: str = layer.id()

    def _on_collect() -> None:
        print(
            f"[memory] layer {layer_id} was garbage-collected. "
            "Ensure it was removed from QgsProject before release."
        )

    weakref.finalize(layer, _on_collect)

Pitfalls and Debugging #

• RuntimeError: wrapped C/C++ object ... has been deleted — Root cause: Python holds a wrapper after the C++ owner (e.g. QgsProject) destroyed the underlying object. Fix: always call sip.isdeleted(obj) before dereferencing objects whose C++ lifetime is externally managed. Never store raw layer references across signal boundaries without a sip.isdeleted() guard.

• Resident-set growth that never levels off — Root cause: QgsFeatureIterator left open inside a for loop that raised an exception before del features ran. Fix: always wrap the iterator in a try/finally block so the del runs even on exception paths.

• Memory usage spikes during large-scale reprojection — Root cause: a single QgsCoordinateTransformContext accumulates PROJ datum-shift grids and pipeline descriptors for every CRS pair it encounters. Fix: delete and recreate the context every N transforms as shown in the batch script pattern above.

• Lambda signal connections preventing collection — Root cause: QgsProject.instance().someSignal.connect(lambda: self.method()) captures self strongly inside the lambda closure, keeping self alive as long as the project exists. Fix: use weakref.ref as shown in the LayerMonitor pattern; or disconnect explicitly in the class’s __del__ or cleanup() method.

• gc.collect() appears to have no effect on memory — Root cause: Python’s GC resolves Python-level reference cycles but cannot free C++ heap memory. Fix: ensure the SIP wrapper itself is deleted (del obj) so SIP calls the C++ destructor; only then does the native heap shrink.

• Silent data corruption when re-using a deleted layer — Root cause: sip.isdeleted() returns True but Python code proceeds anyway (logic error). Fix: treat a True result from sip.isdeleted() as a fatal condition for that code path — return early or raise a domain-specific exception.

• tracemalloc shows steady growth in qgis/core allocations — Root cause: QgsRasterBlock objects held across tile iterations. Fix: delete each block inside finally immediately after processing its data, as shown in the raster block pattern. If growth continues, profile with tracemalloc.take_snapshot() and filter by filename to isolate the offending call site.

Diagnostics Workflow #

Use these tools in order — they operate at different memory layers.

import tracemalloc
import gc
import sip

# 1. Start Python-level allocation tracking before the suspect code path.
tracemalloc.start(25)  # 25 frames of traceback depth

# 2. Run the suspect code — e.g. a single batch iteration.
# ... your processing code ...

# 3. Snapshot and inspect the top allocators.
snapshot = tracemalloc.take_snapshot()
top = snapshot.statistics("lineno")
for stat in top[:15]:
    print(stat)

# 4. Force collection and re-snapshot to see what survived.
gc.collect()
snapshot2 = tracemalloc.take_snapshot()
diff = snapshot2.compare_to(snapshot, "lineno")
for stat in diff[:10]:
    print(stat)

tracemalloc.stop()

For native-heap growth that tracemalloc does not capture, use the QGIS Developer Cookbook guidance on enabling QGIS’s internal memory logging, or run the script under valgrind --tool=massif against a standalone QGIS Python runner.

Conclusion #

Controlling memory in PyQGIS means respecting a dual-ownership model: Python’s reference-counting garbage collector operates above the SIP boundary, while QGIS’s C++ parent-child tree controls native heap lifetime below it. The patterns in this page — declaring ownership at instantiation, scoping iterators inside try/finally, periodically resetting QgsCoordinateTransformContext, using sip.isdeleted() guards, and wiring weakref into signal connections — cover the failure modes responsible for the vast majority of leaks and crashes seen in production PyQGIS deployments. Apply them consistently across every long-running script, processing algorithm, and plugin class, and validate with tracemalloc and gc profiling before shipping.

Related

• Preventing memory leaks when processing large GeoTIFF rasters — deep dive into QgsRasterBlock tiling and sub-extent iteration
• Vector and raster data access patterns — provider architecture and iterator performance
• Working with QgsProject and the layer registry — ownership lifecycle for registered layers
• Coordinate transformations and CRS handling — QgsCoordinateTransform context management
• Signal and slot event handling in QGIS — strong vs. weak reference patterns in Qt signal connections