SimpleVecDB supports at-rest encryption for both SQLite metadata and usearch index files.

Overview

  • SQLite encryption: Uses SQLCipher for transparent page-level AES-256 encryption with hardware acceleration (AES-NI)
  • Index file encryption: Uses AES-256-GCM to encrypt usearch HNSW index files
  • Zero runtime overhead: Index files are decrypted on load and encrypted on save; search operations have no crypto overhead

Installation

pip install simplevecdb[encryption]

This installs: - sqlcipher3-binary - SQLCipher Python bindings - cryptography - AES-GCM implementation

Basic Usage

from simplevecdb import VectorDB

# Create encrypted database with a passphrase
db = VectorDB("secure.db", encryption_key="my-secret-passphrase")

# Use normally - encryption is transparent
collection = db.collection("documents")
collection.add_texts(
    ["Confidential document content"],
    embeddings=[[0.1] * 384]
)

# Save encrypts the index file
db.save()
db.close()

# Reopen with the same key
db = VectorDB("secure.db", encryption_key="my-secret-passphrase")
results = db.collection("documents").similarity_search([0.1] * 384, k=5)

Key Management

Passphrase vs Raw Key

# Option 1: Passphrase (string) - internally derived to 32-byte key
db = VectorDB("secure.db", encryption_key="my-secret-passphrase")

# Option 2: Raw 32-byte key (more secure, use with a key management system)
import os
raw_key = os.urandom(32)  # Generate once, store securely
db = VectorDB("secure.db", encryption_key=raw_key)

Best Practices

  1. Never hardcode keys - Use environment variables or a secrets manager
  2. Use strong passphrases - At least 20 characters with mixed case/numbers/symbols
  3. Backup your key - If you lose the key, the data is unrecoverable
  4. Consider key rotation - Re-encrypt periodically for compliance
import os
from simplevecdb import VectorDB

# Load key from environment
encryption_key = os.environ.get("SIMPLEVECDB_KEY")
if not encryption_key:
    raise ValueError("SIMPLEVECDB_KEY environment variable not set")

db = VectorDB("secure.db", encryption_key=encryption_key)

Storage Layout

With encryption enabled, files are stored as:

mydb.db                          # SQLCipher encrypted SQLite database
mydb.db.salt                     # 16-byte random salt sidecar (mode 0o600)
mydb.db.default.usearch.enc      # AES-256-GCM encrypted usearch index (v1)
mydb.db.default.usearch.enc.salt # 16-byte salt sidecar for the index

When opened, the index is decrypted to memory (or a temp file). On save() or close(), the index is re-encrypted.

Per-DB random salt (2.6.0+)

Each encrypted database and each encrypted index file gets its own random 16-byte salt, written to a sibling .salt file with mode 0o600. The salt is the second input to PBKDF2-HMAC-SHA256, so two databases that share the same passphrase derive different keys.

The sidecar is created with O_CREAT | O_EXCL so two processes opening the same fresh database concurrently cannot race to write conflicting salts; the loser reads the winner's salt and proceeds. An existing sidecar is never overwritten — clobbering it would render the database permanently unreadable with the original passphrase.

Pre-2.6.0 databases continue to open with a fixed legacy salt when no sidecar is present, so existing on-disk data keeps working unchanged.

v1 index file format (2.6.0+)

Index files written by 2.6.0+ start with a 3-byte version header:

magic   = b"SV"     (2 bytes)
version = 0x01      (1 byte)
nonce   = 12 bytes
ciphertext + GCM tag

The header bytes are bound into the AES-GCM associated_data, so any tampering with the magic or version (including a downgrade attempt that strips them) fails authentication on decrypt. Pre-2.6.0 (v0) blobs have no header and continue to decrypt successfully — decrypt_file detects the format automatically.

Atomic durability

encrypt_file and decrypt_file write to a sibling .tmp file, fsync() the data, set mode 0o600, then os.replace() onto the target. The parent directory is also fsynced so the rename itself is durable on POSIX. A crash mid-write leaves only the orphan temp file — the live target is never torn. encrypt_index_file only unlinks the plaintext after the encrypted output is durably on disk, so an interrupted re-encryption never destroys data.

Performance

Search Operations

Encryption has zero overhead during search because: - SQLCipher uses page-level encryption with AES-NI hardware acceleration - Index files are decrypted once on load, then used directly from memory

Load/Save Operations

Operation Overhead
Database open ~10-50ms for key derivation
Index load (10k vectors, 384 dim) ~50-100ms decrypt
Index save ~50-100ms encrypt
Search 0ms (no crypto)

Error Handling

from simplevecdb import VectorDB, EncryptionError, EncryptionUnavailableError

try:
    db = VectorDB("secure.db", encryption_key="wrong-key")
except EncryptionError as e:
    print(f"Failed to open encrypted database: {e}")

try:
    db = VectorDB("secure.db", encryption_key="secret")
except EncryptionUnavailableError:
    print("Install with: pip install simplevecdb[encryption]")

Common Errors

Error Cause Solution
EncryptionUnavailableError Missing dependencies pip install simplevecdb[encryption]
EncryptionError: wrong key Incorrect passphrase Use the correct key
ValueError: In-memory Used :memory: with encryption Use a file path

Limitations

  1. In-memory databases cannot be encrypted - Encryption is for at-rest data
  2. Key cannot be changed - To change keys, export data and re-import
  3. Performance on large indexes - Decryption on load may take several seconds for 100k+ vectors

Security Notes

  • SQLCipher uses AES-256-CBC with HMAC-SHA512 for authentication.
  • Index encryption uses AES-256-GCM with random 96-bit nonces (secrets.token_bytes); each save generates a fresh nonce.
  • Key derivation uses PBKDF2-HMAC-SHA256 with 600,000 iterations (OWASP 2024 recommendation) and a per-DB random salt.
  • v1 file format binds the magic+version header bytes into AES-GCM associated_data, defeating header tampering and downgrade attacks.
  • Derived keys are cached in a bounded LRU (max 64 entries, serialized by a thread lock) so repeat opens within a process avoid the 600k-iter cost without leaking key material in long-running multi-tenant processes.
  • The encryption key is held in memory during database usage.

API Reference

simplevecdb.encryption

Encryption support for SimpleVecDB.

Provides at-rest encryption for both SQLite metadata (via SQLCipher) and usearch index files (via AES-256-GCM).

Design principles: - Zero performance overhead during search operations - Index files are encrypted only at rest (decrypt on load, encrypt on save) - SQLCipher provides transparent page-level encryption with AES-NI acceleration

Usage

from simplevecdb import VectorDB

Enable encryption with a passphrase

db = VectorDB("secure.db", encryption_key="my-secret-passphrase")

Or with raw bytes (32 bytes for AES-256)

db = VectorDB("secure.db", encryption_key=os.urandom(32))

Requirements

Included in the standard install. If missing, reinstall: pip install --force-reinstall simplevecdb

EncryptionError

Bases: Exception

Raised when encryption/decryption fails.

Source code in src/simplevecdb/encryption.py
class EncryptionError(Exception):
    """Raised when encryption/decryption fails."""

    pass

EncryptionUnavailableError

Bases: ImportError

Raised when encryption dependencies are not installed.

Source code in src/simplevecdb/encryption.py
class EncryptionUnavailableError(ImportError):
    """Raised when encryption dependencies are not installed."""

    def __init__(self) -> None:
        super().__init__(
            "Encryption requires sqlcipher3-binary and cryptography. "
            "Reinstall simplevecdb: pip install --force-reinstall simplevecdb"
        )

create_encrypted_connection(path, key, *, check_same_thread=False, timeout=30.0)

Create an encrypted SQLite connection using SQLCipher.

SQLCipher provides transparent AES-256 encryption at the page level, with hardware acceleration on CPUs supporting AES-NI.

Parameters:

Name Type Description Default
path str | Path

Database file path (cannot be ":memory:" for encryption)

required
key str | bytes

Encryption passphrase or 32-byte raw key

required
check_same_thread bool

SQLite thread-safety setting

False
timeout float

Connection timeout in seconds

30.0

Returns:

Type Description
Connection

sqlite3.Connection with encryption enabled

Raises:

Type Description
EncryptionUnavailableError

If sqlcipher3 is not installed

EncryptionError

If encryption setup fails

ValueError

If trying to encrypt an in-memory database

Source code in src/simplevecdb/encryption.py
def create_encrypted_connection(
    path: str | Path,
    key: str | bytes,
    *,
    check_same_thread: bool = False,
    timeout: float = 30.0,
) -> sqlite3.Connection:
    """
    Create an encrypted SQLite connection using SQLCipher.

    SQLCipher provides transparent AES-256 encryption at the page level,
    with hardware acceleration on CPUs supporting AES-NI.

    Args:
        path: Database file path (cannot be ":memory:" for encryption)
        key: Encryption passphrase or 32-byte raw key
        check_same_thread: SQLite thread-safety setting
        timeout: Connection timeout in seconds

    Returns:
        sqlite3.Connection with encryption enabled

    Raises:
        EncryptionUnavailableError: If sqlcipher3 is not installed
        EncryptionError: If encryption setup fails
        ValueError: If trying to encrypt an in-memory database
    """
    path_str = str(path)

    if path_str == ":memory:":
        raise ValueError(
            "In-memory databases cannot be encrypted. "
            "Use a file path for encrypted databases."
        )

    try:
        from sqlcipher3 import dbapi2 as sqlcipher  # type: ignore
    except ImportError:
        raise EncryptionUnavailableError()

    # Resolve the per-DB salt. For a brand-new database, generate and write
    # a random sidecar; for an existing one, prefer the sidecar but fall
    # back to the legacy fixed salt so pre-2.6.0 databases keep opening.
    db_path_obj = Path(path_str)
    is_new_db = not db_path_obj.exists() or db_path_obj.stat().st_size == 0
    salt = _resolve_salt(db_path_obj, create_if_missing=is_new_db)

    try:
        conn = sqlcipher.connect(  # type: ignore[attr-defined]
            path_str,
            check_same_thread=check_same_thread,
            timeout=timeout,
        )

        # Set the encryption key using PRAGMA. PRAGMA arguments cannot be
        # parameterized via ``?`` placeholders, so we must interpolate. To
        # avoid ever interpolating user-supplied passphrase characters into a
        # quoted SQL string (where escape rules are subtle and SQLCipher
        # parsing has surprising edge cases), normalize *every* key path to
        # 32 bytes via ``_normalize_key`` and feed it as ``x'hex'`` raw-key
        # form. The hex output is restricted to ``[0-9a-f]`` so no quoting
        # or escaping is needed.
        normalized_key = _normalize_key(key, salt=salt)
        conn.execute(f"PRAGMA key = \"x'{normalized_key.hex()}'\"")

        # Verify encryption is working by querying cipher_version
        try:
            result = conn.execute("PRAGMA cipher_version").fetchone()
            if result is None:
                raise EncryptionError(
                    "SQLCipher encryption not active. Database may be corrupted "
                    "or key is incorrect."
                )
            _logger.debug("SQLCipher version: %s", result[0])

            # Validate key by actually reading from the database
            # cipher_version only confirms SQLCipher is loaded, not that the key is correct
            # Reading sqlite_master forces decryption and will fail with wrong key
            conn.execute("SELECT count(*) FROM sqlite_master").fetchone()
        except EncryptionError:
            raise
        except Exception as e:
            conn.close()
            raise EncryptionError(
                f"Failed to verify encryption (wrong key?): {e}"
            ) from e

        # Set performance optimizations (same as non-encrypted)
        conn.execute("PRAGMA journal_mode=WAL")
        conn.execute("PRAGMA synchronous=NORMAL")

        return conn  # type: ignore[return-value]

    except EncryptionError:
        raise
    except Exception as e:
        raise EncryptionError(f"Failed to create encrypted connection: {e}") from e

is_database_encrypted(path)

Check if a database file is encrypted.

Attempts to open with standard sqlite3. If it fails with "not a database", the file is likely encrypted.

Parameters:

Name Type Description Default
path str | Path

Path to database file

required

Returns:

Type Description
bool

True if database appears to be encrypted

Source code in src/simplevecdb/encryption.py
def is_database_encrypted(path: str | Path) -> bool:
    """
    Check if a database file is encrypted.

    Attempts to open with standard sqlite3. If it fails with "not a database",
    the file is likely encrypted.

    Args:
        path: Path to database file

    Returns:
        True if database appears to be encrypted
    """
    path = Path(path)
    if not path.exists():
        return False
    # A zero-byte file would cause sqlite3 to create a fresh DB and return
    # False, masking a missing/corrupt database as unencrypted. Treat empty
    # files as not encrypted but also not a real DB; callers that need to
    # distinguish should check existence and size themselves.
    if _is_zero_byte(path):
        return False

    try:
        conn = sqlite3.connect(str(path))
        # Try to read the schema - this will fail if encrypted
        conn.execute("SELECT name FROM sqlite_master LIMIT 1").fetchone()
        conn.close()
        return False
    except sqlite3.DatabaseError as e:
        if "not a database" in str(e).lower() or "encrypted" in str(e).lower():
            return True
        raise

encrypt_file(input_path, output_path, key)

Encrypt a file using AES-256-GCM.

File format: - 12 bytes: nonce - N bytes: ciphertext - 16 bytes: GCM auth tag (appended by cryptography)

Parameters:

Name Type Description Default
input_path Path

Path to plaintext file

required
output_path Path

Path for encrypted output

required
key bytes

32-byte encryption key

required

Raises:

Type Description
EncryptionUnavailableError

If cryptography is not installed

EncryptionError

If encryption fails

Source code in src/simplevecdb/encryption.py
def encrypt_file(
    input_path: Path,
    output_path: Path,
    key: bytes,
) -> None:
    """
    Encrypt a file using AES-256-GCM.

    File format:
    - 12 bytes: nonce
    - N bytes: ciphertext
    - 16 bytes: GCM auth tag (appended by cryptography)

    Args:
        input_path: Path to plaintext file
        output_path: Path for encrypted output
        key: 32-byte encryption key

    Raises:
        EncryptionUnavailableError: If cryptography is not installed
        EncryptionError: If encryption fails
    """
    try:
        from cryptography.hazmat.primitives.ciphers.aead import AESGCM
    except ImportError:
        raise EncryptionUnavailableError()

    try:
        # Read plaintext
        plaintext = input_path.read_bytes()

        # Generate random nonce (MUST be unique per encryption)
        nonce = secrets.token_bytes(AES_NONCE_SIZE)

        # Encrypt with AES-GCM. Bind the v1 header (magic + version) into
        # the GCM AAD so any tampering with the header bytes — including a
        # downgrade attempt that strips the version byte — fails
        # authentication on decrypt.
        header = _ENC_MAGIC + bytes([_ENC_VERSION])
        aesgcm = AESGCM(key)
        ciphertext = aesgcm.encrypt(nonce, plaintext, associated_data=header)

        # Atomically write: header + nonce + ciphertext to a sibling temp
        # file, fsync, restrict permissions, then os.replace() onto the
        # target path. A crash mid-write leaves only the temp file; the
        # target is never torn.
        _atomic_write_bytes(
            output_path, header + nonce + ciphertext, mode=0o600
        )

        _logger.debug(
            "Encrypted %d bytes -> %d bytes",
            len(plaintext),
            len(nonce) + len(ciphertext),
        )

    except Exception as e:
        raise EncryptionError(f"Failed to encrypt file: {e}") from e

decrypt_file(input_path, output_path, key)

Decrypt a file encrypted with encrypt_file().

Parameters:

Name Type Description Default
input_path Path

Path to encrypted file

required
output_path Path

Path for decrypted output

required
key bytes

32-byte encryption key

required

Raises:

Type Description
EncryptionUnavailableError

If cryptography is not installed

EncryptionError

If decryption fails (wrong key, corrupted data)

Source code in src/simplevecdb/encryption.py
def decrypt_file(
    input_path: Path,
    output_path: Path,
    key: bytes,
) -> None:
    """
    Decrypt a file encrypted with encrypt_file().

    Args:
        input_path: Path to encrypted file
        output_path: Path for decrypted output
        key: 32-byte encryption key

    Raises:
        EncryptionUnavailableError: If cryptography is not installed
        EncryptionError: If decryption fails (wrong key, corrupted data)
    """
    try:
        from cryptography.hazmat.primitives.ciphers.aead import AESGCM
    except ImportError:
        raise EncryptionUnavailableError()

    try:
        # Read encrypted data
        data = input_path.read_bytes()

        # Detect format. v1 starts with magic 'SV' + version byte; anything
        # else is treated as v0 (pre-2.6.0) for backwards compatibility.
        # The v1 header is bound into the GCM AAD on encrypt, so any
        # tampering with the magic/version bytes — including a downgrade
        # attempt that strips them — fails authentication here.
        associated_data: bytes | None = None
        if (
            len(data) >= _ENC_HEADER_LEN
            and data[: len(_ENC_MAGIC)] == _ENC_MAGIC
            and data[len(_ENC_MAGIC)] == _ENC_VERSION
        ):
            associated_data = bytes(data[:_ENC_HEADER_LEN])
            data = data[_ENC_HEADER_LEN:]

        if len(data) < AES_NONCE_SIZE + AES_TAG_SIZE:
            raise EncryptionError("Encrypted file too small to be valid")

        # Extract nonce and ciphertext
        nonce = data[:AES_NONCE_SIZE]
        ciphertext = data[AES_NONCE_SIZE:]

        # Decrypt with AES-GCM. associated_data is bound to the v1 header
        # (or None for legacy v0 files where no header was present).
        aesgcm = AESGCM(key)
        plaintext = aesgcm.decrypt(nonce, ciphertext, associated_data=associated_data)

        # Write decrypted data atomically. Plaintext is sensitive — restrict
        # permissions to owner only.
        _atomic_write_bytes(output_path, plaintext, mode=0o600)

        _logger.debug(
            "Decrypted %d bytes -> %d bytes",
            len(data),
            len(plaintext),
        )

    except EncryptionError:
        raise
    except Exception as e:
        # InvalidTag is raised by cryptography when decryption fails (wrong key/corrupted)
        exc_type = type(e).__name__
        exc_msg = str(e).lower()
        if "tag" in exc_type.lower() or "tag" in exc_msg or "authentication" in exc_msg:
            raise EncryptionError(
                "Decryption failed: wrong key or corrupted data"
            ) from e
        raise EncryptionError(f"Failed to decrypt file: {e}") from e