september 30, 2025
unified multimodal database preview
this repository serves as a source of ethically sourced data to support our self-evolving agi framework, nut beta, for use in multimodal applications.
nrutseab is pleased to announce the unified multimodal database, launching alongside nut beta in december 2025. this repository serves as a source of ethically sourced data to support our self-evolving agi framework, nut beta, for use by creators, scientists, and developers in multimodal applications by providing structured access to consented content rather than relying on large-scale, unverified datasets.
the database draws on community-contributed text, images, videos, audio, and simulations to facilitate nut beta’s pre-training and ongoing adaptation. it incorporates provenance tracking inspired by blockchain methods and decentralized storage to promote transparency and accessibility. below, we describe the database’s structure, its contribution to nut beta’s pre-training methodology, and its intended applications.
purpose and vision
the unified multimodal database provides data resources for nut beta, supporting four pre-training areas relevant to multimodal generation (e.g., video synthesis):
scientific world understanding: causal reasoning for physics, chemistry, and biology.
first-person perspective: egocentric processing for immersive applications.
aesthetic and art styles: creative expression across visual and auditory modalities.
fiction vs. scientific world: distinguishing imaginative from empirical content for verifiable outputs.
through curated, consented data, the database helps reduce environmental impact and potential biases, allowing nut beta to adapt from limited inputs via user interactions. this supports a structured approach to ai training, informed by community input.
database structure and features
content types:
text: scientific papers, codebases, and creative narratives from verified sources (e.g., open-access journals, github opt-ins).
images: artworks, photographs, and diagrams, including creator-submitted sketches and public-domain visuals.
videos: first-person clips (e.g., gopro-style activities), scientific simulations (e.g., nasa astrophysics footage), and creative shorts, all consented.
audio: music tracks, ambient sounds, and voice recordings from ethical sources like bandcamp partnerships.
simulations: 2d/3d physics models and virtual environments (e.g., unity-generated scenarios) for training causal reasoning.
key features:
ethical sourcing: all data is opt-in, with explicit creator consent and clear licensing (e.g., creative commons or bespoke agreements).
provenance tracking: blockchain-inspired metadata logs data origins, ensuring transparency and ip compliance.
decentralized storage: hosted on a distributed cloud for scalability and real-time access, supporting nut beta’s 45ms task performance (e.g., code debugging).
dynamic growth: community contributions (e.g., via artist uploads) expand the database, informing nut beta’s updates.
bias mitigation: curated subsets are audited for diversity and fairness, addressing cultural or scientific variations.
scale: initial launch aims for ~10tb of curated data, with a roadmap for 100tb by 2026 through community growth. this scale is designed to complement nut beta’s agi efficiency with focused inputs.
role in nut beta’s pre-training methodology
the unified multimodal database contributes to nut beta’s four pre-training areas, each adapted for multimodal outputs like video generation. below, we outline the database’s support for each area, based on established techniques and ethical considerations.
1. scientific world understanding: causal reasoning and simulation
database role: provides curated scientific texts (e.g., arxiv papers), simulation data (e.g., molecular dynamics from rdkit), and videos (e.g., nasa fluid dynamics clips) to seed causal priors.
methodology:
core injection: symbolic encodings (e.g., newton’s laws via sympy) are distilled from database texts, augmented by simulation data.
self-supervised modeling: nut beta runs neural physics engines, validated against sparse video subsets, using contrastive learning to isolate causal chains.
rlhf refinement: scientist feedback refines predictions, achieving 85% causal fidelity on tasks like ecosystem modeling.
approach: the database’s data enables nut beta to address complex phenomena (e.g., planetary orbits) with limited compute.
metrics: 90% alignment with empirical benchmarks, low generalization error.
impact: supports nut beta in generating scientifically accurate video content, informed by causal data.
2. first-person perspective understanding: immersive egocentric reasoning
database role: supplies consented first-person videos (e.g., gopro daily activities) and synthetic unity simulations for egocentric training.
methodology:
data augmentation: combines real and synthetic clips to diversify scenarios (e.g., cultural activities, motion capture).
egocentric alignment: self-supervised next-frame prediction (ego4d-inspired) trains spatial-temporal dynamics, with actor-critic rl optimizing navigation.
perspective rlhf: community feedback refines viewpoint shifts (e.g., child vs. adult), reducing biases via multimodal cues.
approach: the database facilitates real-time perspective adaptation for immersive applications.
metrics: 95% trajectory consistency, immersion scores (egolife benchmarks).
impact: aids nut beta in producing immersive video outputs, such as first-person narratives or vr experiences.
3. aesthetic and art style understanding: creative generalization
database role: curates art images, music, and videos (e.g., consented bandcamp tracks, artstation visuals) for style taxonomy.
methodology:
style ontology: clusters database content into embeddings (color, texture, composition) for styles like renaissance or cyberpunk.
generative transfer: diffusion-based neural style transfer (nst) recombines aesthetics, augmented for diversity.
creative rlhf: artist feedback refines outputs, with meta-learning enabling style fusion (e.g., visual-audio harmony).
approach: the database supports adaptation to diverse aesthetics while considering ethical factors.
metrics: 92% style fidelity, diversity indices.
impact: assists nut beta in generating stylized videos and audio, supporting creator workflows in the sandbox.
4. fiction vs. scientific world distinction: verifiable outputs
database role: tags content as “scientific” (e.g., empirical simulations) or “fictional” (e.g., sci-fi narratives) for clear boundaries.
methodology:
adversarial training: generator-discriminator pairs detect confabulations, with self-supervised fact-checking against verified sources.
boundary rlhf: user feedback flags inaccuracies, with nut beta outputting confidence scores (hallucination rate <5%).
approach: the database enables boundary enforcement for reliable multimodal generation.
metrics: <5% hallucination rate, 95% discrimination precision.
impact: helps ensure nut beta’s video outputs are verifiable, suitable for scientific and creative uses.
features of the unified multimodal database
the unified multimodal database incorporates:
ethical practices: consent-driven sourcing with provenance tracking to maintain trust.
efficiency: enables learning from targeted inputs, aligning with nut beta’s reduced compute needs compared to larger datasets like those for sora 2.
community involvement: expands through contributions, allowing nut beta to incorporate new data (e.g., aesthetics or scientific updates).
multimodal support: facilitates nut beta’s use of scientific reasoning, perspectives, aesthetics, and verification for outputs like video generation.
at launch, the database will support nut beta in tasks such as grounded sci-fi trailers or vr simulations, within an ethical framework.
up next
join the journey by sending 'nut' to hello[at]nrutseab.com to join the waitlist for beta access.