JSON Document Management in Redis

In modern distributed architectures, Redis has evolved from a simple key-value store into a powerful Document Database through the RedisJSON module. This allows developers to store, update, and query complex nested structures with sub-millisecond latency, bypassing the overhead of full-object serialization.

1. The Security Layer: ACL and Key-Space Jailing

Before interacting with data, we must establish a secure context. Using Redis Access Control Lists (ACLs), we define a specialized user for our DevOps pipeline:

ACL SETUSER devops_api_user +json.set +json.get

Why this configuration?

  • Category Enforcement: By granting +json.set +json.get, we enable Only get ant set operations to native JSON operations.

2. Native Data Manipulation via CLI

RedisJSON stores documents in a binary format (similar to BSON), enabling efficient access to internal fields using JSONPath syntax.

Storing the Document

To initialize our dataset, we use JSON.SET. The $ sign represents the root of the document:

# Clean start
DEL user:profile:data

# JSON.SET <key> <path> <value>
JSON.SET user:profile $ '{"profile": [{"name": "Fausto", "age": 48, "city": "Porto", "skills": ["Python", "Redis", "PostgreSQL", "Jenkins"]},
{"name": "Ana Silva", "age": 28, "city": "Lisboa", "active": true, "skills": ["Java", "Spring"], "orders": [101, 105]},
{"name": "Bruno Costa", "age": 35, "city": "Braga", "active": false, "skills": ["Go", "Docker"], "orders": []},
{"name": "Carla Matos", "age": 22, "city": "Coimbra", "active": true, "skills": ["JavaScript", "React"], "orders": [200]},
{"name": "Diogo Ferro", "age": 40, "city": "Faro", "active": true, "skills": ["Python", "Django", "PostgreSQL"], "orders": [301, 302, 303]},
{"name": "Elena Santos", "age": 29, "city": "Porto", "active": true, "skills": ["Vue.js", "Firebase"], "orders": [450]},
{"name": "Filipe Rosa", "age": 31, "city": "Setúbal", "active": false, "skills": ["C#", "Azure"], "orders": []},
{"name": "Gonçalo Vaz", "age": 26, "city": "Aveiro", "active": true, "skills": ["Ruby", "Rails"], "orders": [501]},
{"name": "Helena Cruz", "age": 45, "city": "Évora", "active": true, "skills": ["PHP", "Laravel", "Redis"], "orders": [601, 605]},
{"name": "Igor Lima", "age": 33, "city": "Funchal", "active": true, "skills": ["Rust", "Wasm"], "orders": [777]}]}'

The Power of JSONPath Querying

One of RedisJSON’s greatest strengths is its ability to perform Partial Retrieval and Server-Side Filtering, reducing network bandwidth.

Few Examples:

Exact Query:

JSON.GET user:profile '$.profile[?(@.name == "Fausto")]'
"[{\"name\":\"Fausto\",\"age\":48,\"city\":\"Porto\",\"skills\":[\"Python\",\"Redis\",\"PostgreSQL\",\"Jenkins\"]}]"

Broad Extraction: JSON.GET user:profile:data $.profile[*].name extracts only the names into a flat list.

JSON.GET user:profile $.profile[*].name
"[\"Fausto\",\"Ana Silva\",\"Bruno Costa\",\"Carla Matos\",\"Diogo Ferro\",\"Elena Santos\",\"Filipe Rosa\",\"Gon\xc3\xa7alo Vaz\",\"Helena Cruz\",\"Igor Lima\"]"



JSON.GET user:profile $.profile[*].name $.profile[*].age $.profile[*].city
"{\"$.profile[*].age\":[48,28,35,22,40,29,31,26,45,33],\"$.profile[*].city\":[\"Porto\",\"Lisboa\",\"Braga\",\"Coimbra\",\"Faro\",\"Porto\",\"Set\xc3\xbabal\",\"Aveiro\",\"\xc3\x89vora\",\"Funchal\"],\"$.profile[*].name\":[\"Fausto\",\"Ana Silva\",\"Bruno Costa\",\"Carla Matos\",\"Diogo Ferro\",\"Elena Santos\",\"Filipe Rosa\",\"Gon\xc3\xa7alo Vaz\",\"Helena Cruz\",\"Igor Lima\"]}"

    Advanced Filtering: JSON.GET user:profile:data '$.profile[?(@.age > 25 && (@.city == "Porto" || @.city == "Lisboa"))]' The query engine evaluates the logical expression inside [?(...)] directly in memory, returning only the objects that satisfy the condition.

      JSON.GET user:profile '$.profile[?(@.age > 25 && (@.city == "Porto" || @.city == "Lisboa"))]'
"[{\"name\":\"Fausto\",\"age\":48,\"city\":\"Porto\",\"skills\":[\"Python\",\"Redis\",\"PostgreSQL\",\"Jenkins\"]},{\"name\":\"Ana Silva\",\"age\":28,\"city\":\"Lisboa\",\"active\":true,\"skills\":[\"Java\",\"Spring\"],\"orders\":[101,105]},{\"name\":\"Elena Santos\",\"age\":29,\"city\":\"Porto\",\"active\":true,\"skills\":[\"Vue.js\",\"Firebase\"],\"orders\":[450]}]"

      3. Programmatic Implementation: The Python Pattern

      In a production environment, we use the redis-py client. The following implementation demonstrates a “Population and Projection” pattern, where we insert data and then project specific fields into a clean dictionary.

      import redis
import json

# Connection configuration
REDIS_HOST = 'redis.devops-db.internal'
REDIS_PORT = 6379
REDIS_USER = 'devops_api_user'
REDIS_PASSWORD = 'Pr4UmfeLGGJEnBu8zX6VgAYux65C0ide'

def manage_profiles():
    try:
        # Client initialization with RBAC
        client = redis.Redis(
            host=REDIS_HOST,
            port=REDIS_PORT,
            username=REDIS_USER,
            password=REDIS_PASSWORD,
            decode_responses=True
        )

        # 1. Dataset definition
        raw_data = {
            "profile": [
                {"name": "Fausto", "age": 48, "city": "Porto", "skills": ["Python", "Redis", "Jenkins"]},
                {"name": "Ana Silva", "age": 28, "city": "Lisboa", "active": True, "skills": ["Java", "Spring"]},
                # ... (other profiles)
            ]
        }

        # 2. Atomic Storage (Key matches the ~user:profile:* ACL pattern)
        client.json().set("user:profile:data", "$", raw_data)
        print("Successfully stored 'user:profile:data' in Redis.")

        # 3. Server-side filtering using JSONPath
        # Fetching users older than 30
        query_path = "$.profile[?(@.age > 30)]"
        matched_profiles = client.json().get("user:profile:data", query_path)

        # 4. Projection: Clean output for downstream consumers
        print("\nFiltered Profiles (Age > 30):")
        if matched_profiles:
            formatted_output = [
                {
                    "name": p.get("name"),
                    "age": p.get("age"),
                    "city": p.get("city")
                }
                for p in matched_profiles
            ]

            for item in formatted_output:
                print(json.dumps(item, ensure_ascii=False))

    except Exception as e:
        print(f"An error occurred: {e}")

if __name__ == "__main__":
    manage_profiles()

      Executing

      python3 json_redis.py
Successfully stored 'user:profile' in Redis.

Filtered Profiles (Age > 30):
{"name": "Fausto", "age": 48, "city": "Porto"}
{"name": "Bruno Costa", "age": 35, "city": "Braga"}
{"name": "Diogo Ferro", "age": 40, "city": "Faro"}
{"name": "Filipe Rosa", "age": 31, "city": "Setúbal"}
{"name": "Helena Cruz", "age": 45, "city": "Évora"}
{"name": "Igor Lima", "age": 33, "city": "Funchal"}

      4. Key Performance Insights

      1. Atomicity: JSON.SET on a specific path (e.g., $.profile[0].age) is atomic. You don’t need to lock the whole document to update one field.
      2. Memory Efficiency: RedisJSON avoids the overhead of converting JSON to strings repeatedly.
      3. Path Complexity: Using recursive descent (..) or complex filters (?()) is powerful but should be used judiciously on very large arrays to maintain low latency.
      Redis ACLs – Quick Overview

      Redis ACLs – Quick Overview

      Implementation of a first example of a Resilient Cache-Aside Layer with Redis and PostgreSQL.

      Implementation of a first example of a Resilient Cache-Aside Layer with Redis and PostgreSQL.