Elasticsearch Types and Query Interfaces

The Elasticsearch Types module serves as the foundational layer for Sophra’s advanced search capabilities, providing a robust set of TypeScript interfaces and utility functions that bridge the gap between Sophra’s data models and Elasticsearch’s query DSL. This module is critical in enabling Sophra’s sophisticated search operations, including vector embedding support for semantic search, multi-index search capabilities, and real-time indexing and updates.

At its core, this module defines the structure of documents as they are represented in Elasticsearch, encapsulating fields such as content, metadata, and AI-generated features like embeddings and evaluation scores. These definitions ensure type safety and consistency across the entire search pipeline, from document indexing to query construction and result processing.

The architectural decisions reflected in this module prioritize flexibility and performance. By providing a comprehensive set of query interfaces, it allows for the construction of complex, highly optimized search queries that can leverage Elasticsearch’s full feature set. This includes support for bool queries, script scoring, and custom aggregations, enabling Sophra to implement advanced ranking algorithms and personalized search experiences.

Performance considerations are evident in the design of the query interfaces and response transformation functions. The module includes utilities for efficient query construction and result parsing, minimizing overhead in the critical path of search operations. This is particularly important for maintaining low latency in high-volume search scenarios.

One of the unique features of this module is its support for AI-enhanced search capabilities. The BaseDocument interface includes fields for embeddings and evaluation scores, allowing for seamless integration of machine learning models in the search process. This enables Sophra to implement cutting-edge search features such as semantic similarity search and automated content quality assessment.

​

Exported Components

export interface BaseDocument {
  id: string;
  title: string;
  content: string;
  abstract: string;
  authors: string[];
  source: string;
  tags: string[];
  metadata: {
    title?: string;
    [key: string]: unknown;
  };
  processing_status: string;
  created_at: string;
  updated_at: string;
  embeddings: number[];
  evaluationScore: {
    actionability: number;
    aggregate: number;
    clarity: number;
    credibility: number;
    relevance: number;
  };
  evaluation_score: {
    actionability: number;
    aggregate: number;
    clarity: number;
    credibility: number;
    relevance: number;
  };
  yearPublished?: number;
  year_published?: number;
  citationCount?: number;
  viewCount?: number;
}

​

Implementation Examples

const searchParams: SearchParams = {
  query: {
    multi_match: {
      query: "machine learning",
      fields: ["title", "content", "abstract"],
      operator: "AND",
      fuzziness: "AUTO"
    }
  },
  size: 10,
  from: 0,
  sort: [{ "created_at": "desc" }],
  aggregations: {
    tags: {
      terms: {
        field: "tags.keyword",
        size: 5
      }
    }
  }
};

const searchOptions: SearchOptions = {
  index: "documents",
  ...searchParams
};

const results = await elasticsearchClient.search(searchOptions);
const transformedResults = transformSearchResponse(results);

​

Sophra Integration Details

The Elasticsearch Types module integrates deeply with Sophra’s search service and analytics engine. It provides the type definitions and query interfaces used throughout the search pipeline:

1. Document Indexing: The BaseDocument interface is used when indexing new documents, ensuring all required fields are present.
2. Query Construction: The SearchParams interface is used by the search service to construct queries based on user input and application logic.
3. Result Processing: The SearchResponse interface and transformSearchResponse function are used to parse and normalize Elasticsearch responses.
4. Analytics: The evaluationScore fields in BaseDocument are used by the analytics engine to track and analyze content quality metrics.

​

Error Handling

The module includes comprehensive error handling strategies:

Error logging is integrated with Sophra’s Winston-based logging system, ensuring all errors are properly captured and monitored.

​

Performance Considerations

The Elasticsearch Types module is optimized for high-performance search operations:

• Efficient query construction with minimal overhead
• Optimized response transformation for large result sets
• Support for pagination and size limits to manage resource utilization

​

Security Implementation

While this module doesn’t directly handle authentication or authorization, it supports Sophra’s security model:

• BaseDocument includes fields for tracking document ownership and access control
• Query interfaces support adding security filters (e.g., user role-based access)
• Response transformation can be extended to apply security checks on returned documents

​

Configuration

The Elasticsearch Types module is configurable through environment variables and runtime options:

ELASTICSEARCH_INDEX_PREFIX=sophra_
ELASTICSEARCH_MAX_RESULT_WINDOW=10000

These configuration options allow for fine-tuning of the Elasticsearch integration to match specific deployment environments and use cases.