Range Reader Recipes¶

This page collects task-oriented recipes for the byte-range RangeReader API of tileverse-storage. For the full container API (listing, writes, copy/move, presigning), see the Storage overview.

What You'll Learn¶

This guide covers everything you need to know to effectively use the Tileverse Range Reader library:

• Installation: How to add the library to your project
• Quick Start: Get up and running with basic examples
• Configuration: Advanced configuration options for optimal performance
• Authentication: Setting up authentication for cloud providers and HTTP sources
• Troubleshooting: Common issues and their solutions

Real-World Use Cases¶

The Tileverse Range Reader library addresses critical needs in modern geospatial and data processing applications:

🌍 Geospatial Data Processing¶

• Cloud Optimized GeoTIFF (COG) reading for satellite imagery analysis
• PMTiles serving for high-performance web mapping
• GeoParquet processing for large-scale vector analytics
• Zarr access for multi-dimensional geospatial datasets
• FlatGeobuf streaming for efficient vector data queries

🏢 Enterprise Applications¶

• Server-side tile rendering without downloading entire datasets
• Microservices architecture with cloud-native data access
• Data pipelines that process specific portions of large files
• Analytics platforms performing random access on cloud-stored data

🚀 Performance-Critical Systems¶

• Real-time mapping applications requiring sub-second response times
• Large-scale batch processing with optimized I/O patterns
• Mobile and embedded systems with bandwidth constraints
• Multi-tenant platforms with shared caching strategies

Target Audience¶

This guide is designed for:

• Java Developers building geospatial or data processing applications
• Application Architects designing cloud-native data access systems
• DevOps Engineers deploying and optimizing data-intensive applications
• Performance Engineers optimizing I/O patterns and caching strategies
• Library Authors building format readers on a common foundation

Getting Started Quickly¶

If you're in a hurry, start with these essentials:

1. Installation - Add the library to your project
2. Quick Start - Basic usage examples
3. Configuration - Performance optimization

Library Overview¶

The Tileverse Range Reader provides a unified interface for reading byte ranges from various sources:

// The core interface - same for all data sources
public interface RangeReader extends Closeable {
    ByteBuffer readRange(long offset, int length);
    int readRange(long offset, int length, ByteBuffer target);
    BatchReadResult readRanges(List<RangeRequest> requests); // batch read, several ranges in one call
    OptionalLong size();
    String getSourceIdentifier();
}

Supported Data Sources¶

Performance Features¶

The library includes several performance optimization features:

• Memory Caching: Fast access to recently used ranges
• Block Alignment: Optimized read patterns for cloud storage
• Concurrent Access: Thread-safe implementations

Common Use Cases¶

PMTiles and Tiled Data¶

Perfect for accessing tile data without loading entire files:

// Read PMTiles header
ByteBuffer header = reader.readRange(0, 127);
header.flip();

// Read specific tiles based on tile index
ByteBuffer tileData = reader.readRange(tileOffset, tileLength);
tileData.flip();

Large File Processing¶

Process large files in chunks:

long fileSize = reader.size().orElseThrow();
int chunkSize = 1024 * 1024; // 1MB chunks

for (long offset = 0; offset < fileSize; offset += chunkSize) {
    int length = (int) Math.min(chunkSize, fileSize - offset);
    ByteBuffer chunk = reader.readRange(offset, length);
    chunk.flip();
    // Process chunk
}

Cloud Data Analysis¶

Efficiently access cloud-stored datasets:

// Read dataset metadata
ByteBuffer metadata = reader.readRange(0, 1024);
metadata.flip();

// Read specific data sections
ByteBuffer section1 = reader.readRange(metadataSize, sectionLength);
section1.flip();
ByteBuffer section2 = reader.readRange(section1Offset, section1Length);
section2.flip();

Performance Considerations¶

Decorator Stacking Order¶

Stack decorators as BlockAligned(regions) -> Caching -> backend, the aligner outermost:

Application
    -> BlockAlignedRangeReader (region-scoped block alignment, outermost)
    -> CachingRangeReader (exact-range cache)
    -> BaseReader (S3, Azure, HTTP, etc.)
    -> Data Source

The aligner expands a request inside a declared region into whole blocks before the call reaches the cache, which then stores block-grain data for that region; everywhere else the cache stores exact ranges.

The inverted order (caching above the aligner) is legal but pointless: the cache would then store expanded blocks under per-request exact keys, and overlapping reads would never share a block.

Read Pattern Guidelines¶

Memory Management¶

The range cache has no per-reader size. Every CachingRangeReader built without an explicit CacheManager shares one in-memory cache, bounded to 20% of the maximum heap by cached bytes, with entries expiring 60 seconds after their last access. Entries are partitioned by the source identifier of the wrapped reader, and clearCache() (or closing the reader) drops one source's entries.

// Default: the shared cache
RangeReader cached = CachingRangeReader.of(delegate);

// Isolated: a cache of its own, with the same 20% budget
CacheManager isolated = CacheManager.newInstance();
RangeReader tenant = CachingRangeReader.builder(delegate)
    .cacheManager(isolated)
    .build();

Keep the number of cache managers small: each one is another full budget. A reader streaming a file once gains nothing from the cache. Use the reader returned by Storage.openRangeReader directly for that workload.

Next Steps¶

Ready to get started? Choose your path:

Need Help?¶

• Check the Troubleshooting guide for common issues
• Review the Quick Start for code examples
• Read the Configuration guide for advanced setup
• Visit our GitHub repository for the latest updates