Modernizing HugeGraph's Query Engine — GSoC 2026 Final Report

01 Overview

The starting point

A dependency upgrade with a security consequence

HugeGraph's query engine still ran on Java 11, TinkerPop 3.5.1 and Groovy 2.5.x. That baseline lacked JDK 17 compatibility and GraphBinary support, missed 26 Gremlin steps, and relied on a SecurityManager-based sandbox affected by CVE-2024-27348.

The project therefore became three connected pieces of work: update the query runtime, move the platform to Java 17, and replace the old sandbox model.

The dependency and platform upgrades are merged, including the move to TinkerPop 3.8.1. Phase 3 is now implementing the first stage of the security sandbox redesign.

Java 11 17

TinkerPop 3.5.1 3.8.1

Groovy 2.5.x 4.0.25

02 The work

Delivery record

Three phases, with the status kept explicit

Each phase has a separate outcome and evidence trail. The dependency and platform work established the Java 17 and TinkerPop 3.8.1 baseline; Phase 3 has moved from design into sandbox implementation.

Phase 01 Merged

TinkerPop 3.7.6 and Groovy 4

Migrated the query runtime from TinkerPop 3.5.1 to 3.7.6 and Groovy 2.5 to 4.0.25. The work covered package and API changes, PBiPredicate, JSR223 classpath behavior, GraphSON serializers, Cypher ID serialization, and HugeCountStep state reuse after clone/reset.

Changed: TinkerPop suite sync, typed GraphSON V1/V2/V3 restoration, serializer and CI fixes.
Verified: 752 core tests plus full Structure and Process suites on memory and RocksDB, with CI coverage across HBase, macOS, PD, Store, HStore and Cluster.

PR #142 ↗ PR #163 ↗

Phase 02 Merged

Java 17 platform migration

Moved all production modules to Java 17 bytecode and updated the surrounding build and runtime surface: four product Dockerfiles, eight CI workflows, startup scripts, and Maven Enforcer rules. The build accepts Java 17 and deliberately rejects JDK 21 for this release line.

Scope: 38 Maven modules plus the Server, PD and Store runtime paths.
Verified: A clean 38-module reactor package with tests skipped, a Java 17 bytecode sample, focused Reflection and launcher regressions, and security/startup wiring checks. Docker and HStore runtime coverage remained CI responsibilities.

PR #183 ↗ PR #194 ↗

Phase 03 Stage 1 in progress

Replace Groovy query execution

The TinkerPop 3.8.1 foundation is merged. Phase 3 is now implementing the first stage of the sandbox redesign: moving external execution from GremlinGroovyScriptEngine to GremlinLangScriptEngine, which accepts Gremlin grammar rather than arbitrary Groovy.

Decision: Route external Gremlin text and Driver Bytecode through parsing and capability checks before execution. Create a fixed-ScriptId internal executor only if a separately reviewed bundled script must temporarily remain.
Acceptance: All external entry points record zero Groovy evaluation or compilation calls. After fixed scripts are converted, the temporary executor is removed so no server-side Groovy execution remains.

PR #197 · TinkerPop 3.8.1 foundation ↗

TinkerPop 3.8.1 is complete; sandbox redesign Stage 1 is in progress.

03 Architecture

Phase 3 security redesign

Replace Groovy query execution with GremlinLang

With the TinkerPop 3.8.1 upgrade complete, Stage 1 now replaces GremlinGroovyScriptEngine on external query paths with GremlinLangScriptEngine. Gremlin text and Driver Bytecode must pass parsing and capability checks before execution. The fixed-script lane below is optional, isolated and temporary; it exists only if a bundled internal script cannot yet be converted.

Current path

External paths converge on Groovy

Swipe to inspect the diagram

Multiple entry points reach GremlinGroovyScriptEngine, exposing arbitrary eval() and compile() behavior.

Target path

GremlinLang replaces external Groovy

Swipe to inspect the diagram

GremlinLangScriptEngine owns external queries and compatible Schema Templates; an internal Groovy lane is created only when a separately reviewed fixed script temporarily requires it.

Security boundaries

No external Groovy. Network, Job, metadata, mounted-file and environment input cannot reach eval() or compile().
GremlinLang is the execution boundary. GremlinLangScriptEngine parses Gremlin text and compatible Schema Templates with the ANTLR grammar, while Driver Bytecode and nested traversals are recursively checked before execution.
Internal scripts stay isolated. Only pre-approved ScriptIds and typed parameters enter the temporary executor; the default allowlist is empty.
Tests observe the boundary. Call probes verify that each external entry point records zero Groovy evaluation or compilation calls.

Implementation route

01 · Complete Upgrade the query stack to TinkerPop 3.8.1
Adapt grammar, GValue, VariableResolver, serializers, Driver and provider APIs, then verify the Java 17 baseline across the full build and CI matrix.
02 · Stage 1 in progress Replace external Groovy execution
Route Gremlin text and Driver Bytecode through GremlinLangScriptEngine, unified request objects, capability checks and zero-call Groovy probes.
03 · Next Remove server-side Groovy
Close the remaining Job, SubgraphStat, gRPC and startup-script paths, then remove any temporary executor and coupled security logic.

04 Evidence

Code and validation

What shipped, and what still needs proof

The merged work is traceable to five pull requests. TinkerPop 3.8.1 compatibility is now verified; the larger performance benchmark and sandbox implementation evidence remain separate work.

#142 TinkerPop 3.7.6 and Groovy 4Dependency migration, API adaptation, serializer fixes, suite sync and CI work. 16 Jul #163 TinkerPop 3.7.6 compatibility fixesCommunity feedback, external validation and compatibility-gap closure. Jul #183 Java 17 migrationBytecode target, CI, images, scripts, Enforcer rules and security wiring. Aug #194 Java 17 support-range follow-upLimit the supported JDK range to Java 17 and align validation, scripts and documentation. 12 Aug #197 TinkerPop 3.8.1 upgradeProvider API adaptation, predicate correctness fixes, 361 Gherkin scenarios and full CI validation. 22 Aug

Open work after this report

01Complete Stage 1: move external Gremlin execution behind GremlinLang capability checks.
02Close the remaining legacy Groovy paths and define their migration behavior.
03Run the formal Twitter-2010 benchmark and evaluate parsing-cache trade-offs.
04Close release metadata and dependency-check findings.

05 Reflection

What changed my approach

What I learned about compatibility, security and upstream work

01

Working with the TinkerPop community changed the project

After opening issue #3069 ↗, I took the sandbox question to the HugeGraph development mailing list and TinkerPop Discord. TinkerPop PMC member Cole Greer explained how providers were already using GremlinLangScriptEngine, where TinkerPop 4.0 was heading, and which dependency fixes were expected in later 3.7 and 3.8 releases. The discussion also led me to widen the compatibility work in PR #163 ↗. I learned to approach upstream maintainers early with a concrete problem, evidence and focused questions; doing so prevented weeks of work on a dead-end design.

02

The security plan had to change

The proposal began with Groovy compile-time whitelisting through @CompileStatic and AST Customizers. The upstream discussion pointed to GremlinLangScriptEngine as the stronger long-term path, changing Phase 3 from “harden Groovy” to “remove external Groovy.”

03

Serializer compatibility needs shape-level tests

Restoring typed GraphSON V1/V2/V3 exposed a subtle Path bug: generic Iterable handling flattened Path labels and objects. Handling Path first and testing the parsed shape directly was more reliable than inferring correctness from token types.

04

A migration test found a production namespace defect

IoGraph tests in PR #163 ↗ showed that HStore's batch path called read-only getGraphId() instead of getGraphIdOrCreate(). New graphs could share reserved ID 65534 as a physical prefix, allowing one graph's truncation to affect another. The bug traced back to a 2025 API split that had missed batch writes.

05

Test filters are part of the compatibility contract

TinkerPop 3.7 changed several semantics. Broad category filters hid too much, so each exclusion was tied back to HugeGraph's declared capabilities. For supportsUniformListValues, the right fix was to correct the feature contract and remove the broad filters.

06

Review documents have to stay current

Phase-scoped design notes, explicit “out of scope” boundaries, and an evidence-backed filter audit kept long reviews manageable. Stale PR descriptions quickly made otherwise correct changes harder to trust, so documentation maintenance became part of the engineering work.

Personal reflection

My GSoC Reflections

What I enjoyed most

I really enjoyed being deeply involved in the development of an open-source project, communicating with my mentors, and working with related communities such as Apache TinkerPop to solve real-world technical problems. Through this process, I experienced firsthand how welcoming and collaborative the open-source community can be. I also benefited greatly from discussions with my mentors, which helped me learn many valuable software development practices and gain practical engineering experience.

What challenged me most

The most challenging part for me was learning how to ensure that the work I contribute is stable, reliable, and ready for real-world use. This involves much more than implementing features—it requires following development standards, carefully reviewing and evaluating changes, writing sufficient tests, and considering potential compatibility and maintenance issues. I realized that releasing a new version or making changes to a widely used open-source project requires a great deal of caution and responsibility. In many ways, the engineering standards can be even more demanding than those of some real-world software projects.

The merged work leaves HugeGraph on a Java 17 and TinkerPop 3.8.1 baseline. Phase 3 is now implementing the first stage of the sandbox redesign, with external Groovy removal kept visible as active engineering work.