Elasticsearch Query Builder

The Elasticsearch Query Builder is a critical component within the Sophra system, serving as the bridge between high-level search requirements and low-level Elasticsearch query structures. This module is designed to construct sophisticated search queries that leverage Elasticsearch’s powerful features, including text search, vector search, and hybrid combinations of both. It plays a pivotal role in Sophra’s advanced search capabilities, enabling semantic search, multi-index queries, and real-time relevance adjustments.

Architecturally, the Query Builder is positioned as a core utility within the Cortex subsystem, which handles Sophra’s intelligent data processing and search operations. It interfaces directly with the Search Service, translating abstract search intents into concrete Elasticsearch query objects. This abstraction layer allows for seamless integration of complex search logic across Sophra’s microservices architecture, maintaining a clean separation of concerns between search intent and query execution.

Key design decisions in the Query Builder revolve around flexibility and extensibility. The module employs a functional approach, with each query type encapsulated in its own builder function. This design facilitates easy composition of complex queries and allows for future expansion of query types without significant refactoring. The use of TypeScript ensures type safety throughout the query construction process, reducing runtime errors and improving developer experience.

Performance considerations are at the forefront of the Query Builder’s implementation. By constructing queries that leverage Elasticsearch’s native query DSL, the module ensures optimal execution on the Elasticsearch cluster. The hybrid query builder, in particular, employs function scoring to balance text and vector search results, allowing for fine-tuned relevance adjustments without sacrificing performance.

The Query Builder showcases several unique technical capabilities, including support for fuzzy matching in text queries, cosine similarity calculations in vector queries, and weighted combinations in hybrid queries. These features enable Sophra to perform nuanced searches that account for typos, semantic similarity, and multi-faceted relevance criteria, positioning the system at the cutting edge of enterprise search technology.

​

Exported Components

interface TextQuery {
  query: string;              // The text to search for
  fields?: string[];          // Fields to search in (default: ["title^2", "content", "abstract"])
  operator?: "AND" | "OR";    // How to combine multiple terms (default: "OR")
  fuzziness?: string | number; // Tolerance for typos (default: "AUTO")
}

​

buildTextQuery

function buildTextQuery(textQuery?: TextQuery): estypes.QueryDslQueryContainer

Constructs an Elasticsearch query for text-based search. Returns a match_all query if no textQuery is provided.

​

buildVectorQuery

function buildVectorQuery(vectorQuery?: VectorQuery): estypes.QueryDslQueryContainer

Builds a vector similarity search query. Throws an error if vectorQuery is not provided.

​

buildHybridQuery

function buildHybridQuery(
  textQuery?: TextQuery,
  vectorQuery?: VectorQuery,
  boost?: { textWeight?: number; vectorWeight?: number }
): estypes.QueryDslQueryContainer

Creates a combined query using both text and vector search methods. Allows for custom weighting of each search type.

​

Implementation Examples

const textQuery: TextQuery = {
  query: "machine learning",
  fields: ["title^3", "content"],
  operator: "AND",
  fuzziness: 1
};

const esQuery = buildTextQuery(textQuery);

​

Sophra Integration Details

The Query Builder integrates tightly with Sophra’s Search Service, which handles the execution of queries against the Elasticsearch cluster. The typical flow involves:

1. The Search Service receives a search request from the API Gateway.
2. It constructs the appropriate query using the Query Builder.
3. The query is executed against Elasticsearch.
4. Results are processed and enhanced by the ML Pipeline if necessary.
5. Final results are cached in Redis and returned to the client.

​

Error Handling

The Query Builder implements robust error handling to ensure system stability:

​

Performance Considerations

The Query Builder is optimized for performance in several ways:

1. Efficient Query Construction: Queries are built using Elasticsearch’s native query DSL, ensuring optimal execution on the cluster.
2. Caching Integration: The Search Service caches query results in Redis, reducing load on Elasticsearch for repeated queries.
3. Balanced Hybrid Searches: The buildHybridQuery function allows for fine-tuned balancing of text and vector search performance.

​

Security Implementation

While the Query Builder itself does not handle authentication or authorization, it integrates with Sophra’s security model:

• Queries are constructed and executed within the authenticated context of the Search Service.
• Field-level security can be implemented by restricting the fields parameter in text queries based on user roles.
• Vector queries are inherently secure as they operate on pre-computed embeddings, preventing injection attacks.

​

Configuration

The Query Builder’s behavior can be fine-tuned through various configuration options:

ELASTICSEARCH_URL=https://elasticsearch:9200
ELASTICSEARCH_API_KEY=your-api-key-here
VECTOR_DIMENSION=384  # Dimension of vector embeddings

These configurations allow for easy adjustment of search behavior across different environments and use cases within the Sophra ecosystem.