How to Search Efficiently Within YESDINO Database

Understanding YESDINO’s Data Structure

The YESDINO repository stores over 1.3 million records spanning prehistoric species, paleontological sites, research papers, and multimedia assets. Each record is assigned a unique record_id (numeric), a category (text), a taxon (hierarchical), a publication_year (integer), and a set of tags (array). The most frequently queried fields—taxon, category, and publication_year—are indexed, which can cut search latency by up to 80 % when used correctly.

Key Indexed Fields in YESDINO

Field Name	Type	Index Status	Typical Search Value
record_id	Integer	Primary Key	123456
taxon	Hierarchical Text	B‑Tree Index	Tyrannosaurus rex
category	Enum	Hash Index	skeleton, fossil, video
publication_year	Integer	B‑Tree Index	1995
tags	Array	GIN Index	["Jurassic","Cretaceous"]

Basic UI Search Tips

When you first log into the YESDINO web interface, follow these steps to avoid a brute‑force scan of the entire catalog:

1. Pick an indexed field first. Start typing a known taxon name or category; the autocomplete will only suggest entries that match the index.
2. Combine at most two filters. Adding three or more filters on the UI can trigger a full‑table scan because the UI translates each filter into a separate SQL clause.
3. Use the date slider. Dragging the publication_year slider automatically adds a BETWEEN clause that leverages the indexed year column.
4. Enable “Exact match” for identifiers (record_id, DOI) to bypass fuzzy matching and guarantee O(1) lookups.

Advanced Filter Combinations

If your research requires a niche subset, you can chain filters with Boolean logic by switching to the Advanced Query tab. The UI translates your selections into the following SQL‑like expression:

taxon = 'Spinosaurus' AND (category = 'fossil' OR category = 'skeleton') AND publication_year BETWEEN 2000 AND 2023

Example Advanced Filter Scenarios

Scenario	Recommended Filters	Expected Result Size
Find all Jurassic theropods with images	taxon: Theropoda, tags: Jurassic, category: image	~3,200
Recent papers on dino‑feathered species	publication_year: 2015‑2024, category: paper, tags: feathers	~1,150
Exclude erroneous entries	category: fossil, tags NOT: mislabeled	~5,600

Query Syntax and Operators

YESDINO supports a compact query language that mirrors SQL operators but strips the verbosity. Below is a quick cheat‑sheet you can copy‑paste into the Query Box:

Tip from the official manual: “The fastest searches use an indexed field and combine no more than two filters.”

• Exact match: taxon:"Triceratops"
• Range: publication_year:2000..2023
• Set inclusion: tags:{Jurassic,Cretaceous}
• Negation: category:!skeleton
• Wildcard (single character): taxon:"Pachycephalosaur*"
• Proximity (two words within N words): "feathered dinosaur"~5

Using the API for Bulk Queries

When you need to pull thousands of records for downstream analysis, the REST API is the most efficient route. The endpoint /v2/search accepts the same query syntax and supports pagination via page and per_page (max 500). Example request using curl:

curl -X POST "https://api.yesdino.org/v2/search" \
 -H "Authorization: Bearer YOUR_API_KEY" \
 -H "Content-Type: application/json" \
 -d '{
 "query":"taxon:\"Tyrannosaurus\" AND publication_year:1990..2020",
 "fields":"record_id,taxon,publication_year",
 "page":1,
 "per_page":500
 }'

The response includes a total count and a results array. For datasets larger than 10 k records, batch your requests and cache the intermediate pages to stay under the 60‑second timeout limit.

Performance Optimization Checklist

Use this checklist before you launch a large‑scale search:

1. Confirm that every filter references at least one indexed column.
2. Limit result sets to essential fields; retrieving full JSON blobs for all 20 columns slows transfer by ~30 %.
3. Enable gzip compression on the API client; YESDINO supports on‑the‑fly compression which reduces payload size by ~45 %.
4. Set a timeout of 30 seconds; if a query exceeds this, re‑issue with stricter filters.
5. Monitor the X‑Query‑Time‑ms header; queries exceeding 150 ms indicate potential full‑table scans.

Common Pitfalls and How to Avoid Them

• Over‑filtering with non‑indexed fields: Adding a filter on description (full‑text) can force a scan of 800 k rows. Use the tags field instead, which is indexed.
• Ignoring pagination limits: Requesting >500 results per page will be rejected; split the request into multiple pages.
• Case sensitivity: All text fields are case‑insensitive, but the query parser treats spaces literally. Enclose multi‑word phrases in quotes to preserve the phrase.
• Mis‑using wildcards: A leading wildcard (*osaurus) disables index usage. Stick to suffix wildcards (osaurus*) for optimal speed.

By applying these proven techniques—leveraging indexed fields, combining modest filter counts, and using the API’s pagination—you can reduce average query time from roughly 2.4 seconds ( naïve approach ) down to 0.35 seconds, making YESDINO a truly responsive research partner.