Motion2MixamoRig transfers human motion from a monocular video—or a pose from a still image—onto an Adobe Mixamo-rigged 3D character. This release checks the finished character for self-intersection, can drive the fingers, and pins planted feet to the floor. ### Highlights - **Self-intersection diagnostics**: every run writes `diagnostics.json` (which parts intersect, on which frames, and how deep). `m2mr diagnose <run> --visualize` also writes `diagnostics.html`, with the source video beside the character, a timeline, and a clickable issue list. In Blender, the Diagnostics box under Progress jumps between problem frames, and **Open HTML Report** builds that page. - **Fingers**: `m2mr run --detect-hands` uses HaMeR to drive the 30 Mixamo finger bones. It is off by default. It needs `MANO_RIGHT.pkl`. The first run downloads about 2.7 GB of hand weights, and the clip takes about twice as long. - **Planted feet**: a planted foot stays where it landed (the ball when the ball is down, the heel only when the toes are up). The sole is then set on the floor. `--no-foot-lock` keeps the raw retarget, so the feet may slide; the sole is still set on the floor. - **Floor markers** on the report and in the Blender viewport, so movement can be judged against the floor. - **Review notes**: in `diagnostics.html`, mark a stretch that still looks wrong. **Download notes** saves those notes with the intersection list for a later pass. - **Documentation**: the README and the user guide are in 24 languages. The add-on UI is in 19, including Italian. The download table lists Windows as coming soon. ### What you will notice - After import, the timeline runs from frame 0 to n−1, the same numbers as `diagnostics.json`. - A person who turned away and back is less likely to stay facing the wrong way for the rest of the clip. - **Open HTML Report** can find a Homebrew `ffmpeg` when Blender was opened from the Dock or Finder, so the source video is included. ### Install the Blender add-on 1. Download `motion2mixamorig-0.1.8-macos-arm64.zip` from the assets below (do not unzip it). This release has a macOS (Apple Silicon) build only. A Windows build is coming soon. 2. In Blender 4.2+: **Edit → Preferences → Get Extensions → ∨ → Install from Disk…** 3. Follow the [User Guide](https://github.com/MaxLiu22/Motion2MixamoRig/blob/v0.1.8/docs/user-guide.md). Updating from v0.1.7: remove the old add-on, quit Blender fully, install this zip, then click **Update Runtime**. Downloaded models are kept. The new checks, the finger pass, and the foot pin live in the runtime, so this step is required. ### Important notes * Tested on macOS (Apple Silicon) only. A Windows build is coming soon. * Input currently supports one clearly visible, full-body person. * The SMPL-X body model and the Mixamo character must be obtained separately under their respective licenses. `--detect-hands` also needs the MANO right-hand model. * Non-commercial use only, because of upstream model licenses (see `NOTICE.md`). Full changelog: [v0.1.7...v0.1.8](https://github.com/MaxLiu22/Motion2MixamoRig/compare/v0.1.7...v0.1.8)
Training generalizable embodied agents requires realistic, varied, and simulation-ready indoor environments. Language-guided methods support semantic planning and object-level reasoning, but precise spatial grounding remains difficult. Visual generative models offer rich priors over plausible arrangements, yet their outputs require geometric reasoning to become physically valid 3D scenes. We introduce SceneBloom, an agentic framework that combines visual layout sampling with contract-guided 3D compilation. A hierarchical sampler uses diffusion priors to generate furniture arrangements within functional zones. A compiler then uses relations extracted from each proposal to guide geometric fitting and local pose correction. Across five room types, SceneBloom attains the highest visual quality and physical stability among the compared methods and is the only method with no detected mesh collisions and out-of-bounds objects. Its layouts are also closer to the distribution of real rooms than those of all compared baselines. Object navigation policies trained in these realistic scenes achieve a higher success rate in held-out real scans than those trained on an equal number of real training scans. Code is available at https://github.com/Tinaliu0123/scenebloom.
Actions that condition world models are highly heterogeneous, and thus existing action-conditioned world models are typically built for individual action types, fragmenting already limited action-annotated data across separate models. We introduce UniAction, a unified action-conditioning framework that represents any action as a sparse set of action elements, each with a discrete semantic identity and a continuous spatiotemporal grounding coordinate. By mapping these coordinates directly into the pretrained video model's native positional space, UniAction allows heterogeneous actions to condition a shared world model within the same spatiotemporal structure. We further introduce cardinality-calibrated attention to address the bias from differing action representation granularities. We evaluate UniAction on four representative action forms: force, camera control, hand pose, and robot pose. UniAction outperforms conditioning alternatives in action following, and a single jointly trained model surpasses the action-specific experts, showing that heterogeneous action data benefit from unified model and joint training.
Most fine-tuning teaches a model to produce a better answer to a single prompt. Agentic tasks are different. An agent that files an expense report or triages a support ticket takes a sequence of actions, and each step depends on what came before. There’s rarely one correct sequence to imitate, and the only reliable feedback is whether the whole sequence succeeded. Supervised fine-tuning has no labels for that, and single-turn reinforcement learning cannot credit a decision made three steps before the outcome. Multi-Turn Reinforcement Learning (MTRL) on Amazon SageMaker AI trains models on exactly that signal. SageMaker AI sends prompts to an agent you host on Amazon Bedrock AgentCore, or on your own infrastructure, such as Amazon Elastic Kubernetes Service (Amazon EKS), Amazon Elastic Compute Cloud (Amazon EC2), or AWS Fargate, through an AWS Lambda forwarder . The example here uses AgentCore. The agent calls the policy model being trained, acts in your environment over as many turns as the task needs, and reports one reward when the episode ends. You don’t specify the right trajectory, only how to score the result. This post uses Wordle , a game in which a player has six tries to guess a hidden five-letter word, as the example. After each guess, every letter turns green if it is in the right position, yellow if it’s in the word but in a different position, and gray if it isn’t in the word. The game suits MTRL because the rules are few, each guess depends on the feedback from the last, and the outcome is scored objectively. Wordle is a teaching example, and frontier models already play it well. In practice, you would use MTRL to make a small, low-cost model good at a multi-step task specific to your business, such as an internal workflow that public models were never trained on. The same pipeline applies to that setting. In our run, MTRL took gpt-oss-20b from a 61 to an 84 percent solve rate on held-out words at its default reasoning effort, and from 15 to 69 percent at the low effort it trained at, for about $339 in training tokens. In this post, we walk through the pipeline end-to-end: deploying the agent and reward to AgentCore, preparing the dataset, training, monitoring in MLflow, and evaluating at serving settings. We also share the reward-design mistakes and training failures we hit. Your datasets, agent, and trained model stay in your AWS account and are not used to train the base models . All code is in the accompanying aws-samples repository . Solution overview The system has two halves. SageMaker AI owns the training job and the policy model. AgentCore hosts the agent, the Wordle environment, and the reward function. The following diagram shows how they connect. Figure 1: MTRL on SageMaker AI, where the training job invokes the AgentCore-hosted agent once per rollout to sample the policy model, play the game, and report a reward For every rollout, the MTRL service invokes the AgentCore runtime, a capability of Amazon Bedrock AgentCore, with one training row, which holds one secret word. A Strands agent in the runtime calls the policy model through the SageMaker Job Runtime’s OpenAI-compatible endpoint. The model plays up to six guesses through one tool, guess_word , and after each guess the environment returns per-letter feedback and a summary of what is known. When the game ends, the agent returns one reward. Group Relative Policy Optimization (GRPO) uses the spread of rewards across several rollouts of the same word to update a Low-Rank Adaptation ( LoRA ) adapter on the policy model. Three design points matter for what follows: The agent and reward live in one place. A single Python file deployed to AgentCore holds the environment, the tool, and the scoring function. The dataset carries only the secret word. The word lists bundled with the agent are built from the open-source SCOWL word lists and are used only for scoring. The reward is outcome-dominant. A solve pays 1.05–1.55 depending on speed, a loss pays -0.5, and never making a valid guess pays -1.5. A small shaping term rewards good strategy within each tier. Supported models and AWS Regions MTRL is serverless: you pick a base model and the service provisions the training and sampling infrastructure. At the time of writing, it supports the following base models, each trained with a LoRA adapter: Base model AWS Regions Amazon Nova 2 Lite US East (N. Virginia) (us-east-1), US West (Oregon) (us-west-2) GPT-OSS-20B US East (N. Virginia) (us-east-1), US West (Oregon) (us-west-2) Gemma 4 31B (instruction-tuned) US West (Oregon) (us-west-2) Qwen 3.6 27B US West (Oregon) (us-west-2) We chose GPT-OSS-20B because it has the lowest per-token price of the four and its reasoning effort is controllable per request. See the MTRL documentation for the current list. Pricing You don’t choose instances or size a cluster. You pay per million tokens in three dimensions, priced by base model: Prefill: input tokens the policy model processes. These accumulate in a multi-turn task, because every turn re-reads the history. Sample: tokens the policy model generates, including reasoning tokens. Train: tokens processed in the backward pass that updates the LoRA adapter. The 100-step run consumed 491.6 million prefill, 159.7 million sample, and 643.3 million train tokens across 105,486 rollouts in 12 hours 40 minutes. At GPT-OSS-20B list prices in the US East (N. Virginia) Region (us-east-1) at the time of writing, that is about $339: Dimension Tokens Price per 1M Cost Prefill 491,616,806 $0.12 $58.99 Sample 159,737,098 $0.30 $47.92 Train 643,308,554 $0.36 $231.59 Total 1,294,662,458 $338.50 The train dimension dominates and scales with trajectory length, so shorter games and less verbose reasoning cut the bill. The AgentCore runtime bills separately for the vCPU and memory the agent uses. For this run, that was about $79, from the runtime’s Amazon CloudWatch usage metrics at us-east-1 list prices. DescribeJob reports the token counts under BillableTokenUsage . See Amazon SageMaker AI pricing for current rates. Prerequisites To follow this walkthrough, you need the following. See also the MTRL prerequisites . Python 3.10 or later, uv, Node 20 or later, and the AgentCore CLI ( npm install -g @aws/agentcore ). See Get started with Amazon Bedrock AgentCore for installation and the IAM permissions the CLI needs. botocore 1.43.37 or later, on the machine that launches the job. This is the first version that includes the AgentRFT CreateJob operation. Two AWS Identity and Access Management (IAM) roles, described in Step 1. An Amazon Simple Storage Service (Amazon S3) bucket in the same Region as the agent. The training job must also run in that Region, because CreateJob rejects a cross-Region runtime ARN. Step 1: Create the IAM roles MTRL uses two roles, and one of them has a trust policy that the general SageMaker AI documentation doesn’t mention. Agent runtime role. agentcore deploy creates this role for you. It trusts bedrock-agentcore.amazonaws.com and needs the AmazonSageMakerJobRuntimeAccess managed policy, which lets the agent call the policy model and report rewards. The CDK stack in the sample repository attaches this policy automatically. Training job role. You create this one. It must trust job.sagemaker.amazonaws.com with both sts:AssumeRole and sts:TagSession , and it needs the AmazonSageMakerJobFullAccess managed policy. { "Version": "2012-10-17", "Statement": [{ "Effect": "Allow", "Principal": {"Service": "job.sagemaker.amazonaws.com"}, "Action": ["sts:AssumeRole", "sts:TagSession"] }] } Note: The trust principal is job.sagemaker.amazonaws.com , not sagemaker.amazonaws.com . With the generic principal, CreateJob fails with “Could not assume role,” and the message names the correct principal only after retries. Step 2: Deploy the agent to Amazon Bedrock AgentCore Clone the sample repository , set your account and Region in Wordle/agentcore/ aws-targets.json , and deploy: git clone https://github.com/aws-samples/sample-multi-turn-rl-wordle cd sample-multi-turn-rl-wordle/Wordle agentcore validate agentcore deploy The CLI builds the agent’s dependencies for the runtime’s Linux ARM64 platform, creates the execution role with AmazonSageMakerJobRuntimeAccess attached, and deploys through AWS CloudFormation. We recommend it over uploading a code package on the console, where it’s common to get the roughly 700 MB of compiled dependencies or the policy attachment wrong. Note the runtime ARN in the output for the training job. Figure 2: The Wordle agent runtime in the Amazon Bedrock AgentCore console with status Ready The runtime reads two environment variables from agentcore.json . REASONING_EFFORT=low sets reasoning_effort on every request to the policy model. In our evaluation jobs, low effort used 63 to 83 percent fewer sample tokens per game than medium. REASONING_PROMPT_HINT=true also adds “Reasoning: low” to the system prompt, which is how gpt-oss reads the setting in its Harmony chat format. Neither variable gives the model any hint about the secret word. Figure 3: Runtime environment variables showing REASONING_EFFORT set to low The agent is one file, main.py . Simplified, its entrypoint looks like the following. For your own task, you change the task parsing, the tools, and the reward: app = BedrockAgentCoreApp() @app.entrypoint # outermost: starts the HTTP server AgentCore invokes @sagemaker_rft_handler # reports the returned reward to SageMaker AI def handle_rollout(payload): prompt, answer = parse_task(payload["prompt"]) game = WordleGame(answer) @tool def guess_word(guess: str) -> str: """Submit a 5-letter Wordle guess and get feedback.""" return game.guess(guess) agent = Agent(model=build_model(), system_prompt=SYSTEM_PROMPT, tools=[guess_word]) agent(prompt) return {"reward": game.final_reward()} Important: The sagemaker_rft_handler decorator does not start an HTTP server by itself. Wrap it with BedrockAgentCoreApp and @app.entrypoint , with the entrypoint decorator outermost. Without this, every invocation times out after 30 seconds. Step 3: Verify the runtime Before spending GPU time, confirm the whole chain works with a payload shaped like the one the training service sends. The repository includes one as Wordle/ payload.json : { "prompt": "{\"prompt\": \"Guess the 5-letter word\", \"answer\": \"beach\", \"id\": \"smoke-test\"}", "metadata": { "jobArn": "arn:aws:sagemaker:us-east-1:111122223333:job/AgentRFT/smoke-test", "rolloutId": "r1" } } The fake job ARN is expected to fail, and the way it fails is the proof. Put your own account ID into it first: for a job ARN in another account, the service returns a 403 before it checks whether the job exists, which looks like a broken IAM setup. From the Wordle directory, run: ACCOUNT=$(aws sts get-caller-identity --query Account --output text) sed "s/111122223333/$ACCOUNT/" payload.json > /tmp/payload.json aws bedrock-agentcore invoke-agent-runtime \ --agent-runtime-arn <RUNTIME_ARN> \ --runtime-session-id smoke-test-$(date +%s)000000000000000 \ --payload fileb:///tmp/payload.json /dev/stdout In the runtime’s Amazon CloudWatch logs, you see the agent parse the prompt, authenticate to the Job Runtime endpoint, and receive a 400 error reading “Could not find job.” That failure confirms that parsing, authentication, and error reporting work. The command itself prints {“reward”: -1.5}, because the agent never reached the policy model. A 403 reading “not authorized to access this resource” means the job ARN still names another account. A 30-second timeout instead means the HTTP server is not starting (see the Important note in Step 2). Step 4: Prepare the dataset The MTRL dataset has no target responses, transcripts, or labels. Each row is one task for the agent to attempt. Here are the first three rows of the training file: {"prompt": "{\"prompt\": \"Guess the 5-letter word\", \"answer\": \"riser\", \"id\": \"wordle_train_0000\"}"} {"prompt": "{\"prompt\": \"Guess the 5-lett
The Global Performance Tuner is a feature for automatic performance tuning released in NVIDIA TensorRT 11.1. It provides automatic, accuracy-aware and... The Global Performance Tuner is a feature for automatic performance tuning released in NVIDIA TensorRT 11.1. It provides automatic, accuracy-aware and end-to-end performance tuning within TensorRT by customizing different optimizations for different models. For more details, see Part 1 of this series, which introduces the challenges associated with scheduling (a critical stage in deep learning… Source
Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For... Deploying a trained neural network involves more than translating each operator into a GPU kernel. It is a compilation problem with many legitimate answers. For example, when NVIDIA TensorRT builds an engine, it makes interdependent decisions about graph-IR transformations, fusion, implementation selection, memory placement, and quantization. This process is called scheduling. Source
Longitudinal causal effect estimation is complicated by time-varying confounding and treatment-confounder feedback loops. Existing methods generally require data-specific nuisance-model fitting, and many are tailored to prespecified treatment sequences or prediction horizons, leading to time-consuming computation at deployment. We introduce *Policy In-Context Longitudinal causal Effect estimation with Prior-data Fitted Networks* (**PICLE-PFN**), the first foundation model for longitudinal causal effect estimation that supports *dynamic* treatment policies. PICLE-PFN represents target policies through in-context history-decision demonstrations, providing a unified interface for a broad class of treatment policies. Pretrained over synthetic data sampled from longitudinal structural causal models, PICLE-PFN amortizes causal effect estimation across datasets, policies, and horizons. Given a new observational dataset and target policy, it directly predicts conditional average potential outcomes (CAPOs) in one forward pass without task-specific training or hyperparameter tuning. Across synthetic and semi-synthetic longitudinal benchmarks, PICLE-PFN achieves competitive RMSE and substantially improved uncertainty calibration with a fraction of deployment runtime, compared to state-of-the-art conventional estimators trained and tuned separately for each dataset.
Updating the Amazon Sagemaker AI real-time endpoint configuration (changing the instance type, instance count, or environment variables) causes deployments through AWS CloudFormation or the AWS Cloud Development Kit (AWS CDK) to fail. SageMaker AI requires you to deregister the scalable target before updating the endpoint, but when you deregister it, CloudFormation can’t track the AWS Application Auto Scaling resources. In this post, we show you how to use a reusable CDK construct with AWS CloudFormation custom resources to orchestrate deregister and register auto scaling automatically during endpoint updates. You can update your endpoint configuration and auto scaling parameters in a single deployment without manual intervention. Why this problem occurs When you deploy a SageMaker AI endpoint with native CloudFormation auto scaling resources, then change the endpoint configuration and redeploy, several failure modes emerge. The specific failure depends on what you change and how you work around the issue. Changing the endpoint configuration with auto scaling registered CloudFormation tries to update the endpoint while the scalable target is still registered. SageMaker AI rejects the update with the following error: The variant(s) "[AllTraffic]" must be deregistered as scalable targets with Application Auto Scaling before they can be removed or have their instance type updated. SageMaker AI blocks the endpoint update before CloudFormation reaches the auto scaling resources. Manually deregistering auto scaling, then changing only the endpoint configuration If you deregister the scalable target through the AWS Command Line Interface (AWS CLI) and redeploy with only an endpoint configuration change, the deployment succeeds. However, CloudFormation doesn’t register auto scaling even though it is in the CloudFormation template. Because the auto scaling parameters in the CloudFormation template haven’t changed, CloudFormation doesn’t register the scalable target or re-apply the scaling policy. The endpoint is now running without auto scaling, and there is no error to indicate anything went wrong. Manually deregistering auto scaling, then changing auto scaling parameters If you deregister the scalable target through the AWS CLI and redeploy with auto scaling parameter changes, CloudFormation tries to update the ScalableTarget resource that no longer exists: Resource of type 'AWS::ApplicationAutoScaling::ScalableTarget' with identifier 'endpoint/<name>/variant/AllTraffic|sagemaker:variant:DesiredInstanceCount|sagemaker' was not found. This failure is notable because the rollback also fails (UPDATE_ROLLBACK_FAILED), leaving the stack in a broken state that requires manual intervention in the CloudFormation console to recover. Solution overview You can use this CDK construct to replace the native AWS::ApplicationAutoScaling::ScalableTarget and AWS::ApplicationAutoScaling::ScalingPolicy CloudFormation resources with three custom resources. These custom resources deregister and register auto scaling around the endpoint update with explicit dependency ordering, enforced by node.add_dependency calls in CDK. The full sequence is: Deregister (custom resource): Removes the scalable target before the endpoint update begins. On first deploy, no scalable target exists yet, so the handler handles the error gracefully and continues. Endpoint update (CDK-managed): CDK updates the endpoint resource normally, swapping to the new endpoint configuration. Register Scalable Target (custom resource): Waits for the endpoint to reach InService status, then registers the scalable target. Attach Scaling Policy (custom resource): Waits for the scalable target to be queryable, then applies the target tracking policy. Figure 1: Custom resource orchestration of SageMaker AI endpoint auto scaling around a CDK-managed endpoint update. Register Scalable Target and Attach Scaling Policy use the CDK provider’s asynchronous completion pattern. Each custom resource has an on_event handler that returns immediately and an is_complete handler. The provider’s waiter re-invokes the is_complete handler on an interval (every 30 seconds) until its readiness check passes. The provider’s total_timeout (1 hour for Register Scalable Target, 15 minutes for Attach Scaling Policy) bounds the wait, so it isn’t limited by the timeout of any single Lambda invocation. A ConfigHash property on each custom resource forces CloudFormation to re-invoke the Lambda functions whenever a configuration value changes. The endpoint configuration and auto scaling parameters are defined together because a single ConfigHash drives the auto scaling orchestration. While the construct defines three custom resources, the deployed stack contains more than three Lambda functions. Each custom resource is backed by a CDK cr.Provider , and the provider framework generates its own helper Lambda functions. A synchronous provider (Deregister) adds one framework function. An asynchronous provider (Register Scalable Target and Attach Scaling Policy) adds three ( onEvent , isComplete , and onTimeout ) plus an AWS Step Functions state machine that drives the waiter. As a result, the deployed stack contains three custom resources, two Step Functions state machines, and twelve Lambda functions. Five of the resources hold the construct’s own handler code (one for Deregister, and an on_event and is_complete pair for each asynchronous resource), with the remaining seven generated by the provider framework. You don’t write or maintain the framework-generated functions, but they are present in the stack and worth accounting for when you review the deployed resources. Walkthrough In this walkthrough, you deploy a SageMaker AI endpoint with auto scaling using the provided example stack, then verify that configuration updates work correctly. The walkthrough includes the following steps: Clone the repository and install dependencies. Configure your endpoint and auto scaling settings. Deploy and verify. Test an update. The source code is available in the linked GitHub repository . Prerequisites For this walkthrough, you need the following: An AWS account with permissions for Amazon SageMaker AI, AWS Lambda, AWS Identity and Access Management (IAM), Application Auto Scaling, and Amazon CloudWatch. AWS CDK v2 installed. Python 3.12 or later. uv for Python dependency management. A SageMaker AI model artifact in Amazon Simple Storage Service (Amazon S3) and a container image in Amazon Elastic Container Registry (Amazon ECR), or a SageMaker AI built-in algorithm container. Clone the repository and install dependencies Clone the repository: git clone https://github.com/aws-samples/sample-sagemaker-endpoint-autoscaling.git cd sample-sagemaker-endpoint-autoscaling Install dependencies: uv sync Configure your endpoint and auto scaling settings The endpoint and auto scaling settings are defined together in config/ config.yaml because a single config_hash drives the auto scaling orchestration. A change to either section triggers the full sequence. Edit config/ config.yaml with your values: endpoint: endpoint_name: "my-model-endpoint" image: "123456789012.dkr.ecr.us-east-1.amazonaws.com/my-model:latest" model_data_url: "s3://my-bucket/models/model.tar.gz" instance_type: "ml.m5.large" initial_instance_count: 1 environment: {} autoscaling: min_capacity: 1 max_capacity: 10 target_value: 100 scale_in_cooldown: 600 scale_out_cooldown: 120 The endpoint parameters map to the SageMaker AI CreateModel and CreateEndpointConfig APIs. The auto scaling parameters map to the Application Auto Scaling RegisterScalableTarget and PutScalingPolicy APIs. Deploy and verify Bootstrap your AWS environment if you haven’t already: uv run cdk bootstrap Deploy the stack: uv run cdk deploy InferenceStack CloudFormation creates the SageMaker AI model, endpoint configuration, and endpoint, then runs the three custom resources in sequence to register auto scaling. Verify that the deployment registered the scalable target: aws application-autoscaling describe-scalable-targets --service-namespace sagemaker --resource-ids endpoint/my-model-endpoint/variant/AllTraffic Verify that the deployment applied the scaling policy: aws application-autoscaling describe-scaling-policies --service-namespace sagemaker --resource-id endpoint/my-model-endpoint/variant/AllTraffic Test an update Change a value in config/ config.yaml , for example, update the instance type from ml.m5.large to ml.m5.xlarge , and redeploy: uv run cdk deploy InferenceStack After the deployment completes, verify that auto scaling is still registered and has the updated values: aws application-autoscaling describe-scalable-targets --service-namespace sagemaker --resource-ids endpoint/my-model-endpoint/variant/AllTraffic Integrating with an existing stack Use the example stack if you’re starting fresh or want a reference implementation. Use the construct directly if you already have a CDK stack with SageMaker AI resources. To integrate with an existing stack, copy the sagemaker_autoscaling/constructs/ directory into your project and add the construct alongside your endpoint resources: import json from sagemaker_autoscaling.constructs.sagemaker_autoscaling_construct import ( AutoscalingConfig, SagemakerAutoscalingConstruct, SagemakerAutoscalingProps, ) # Both endpoint and auto scaling values must be in config_hash so that # a change to either triggers the auto scaling orchestration. config_hash = json.dumps({ "endpoint": { "image": "123456789012.dkr.ecr.us-east-1.amazonaws.com/my-model:latest", "modelDataUrl": "s3://my-bucket/models/model.tar.gz", "instanceType": "ml.m5.large", "initialInstanceCount": 1, }, "autoscaling": { "minCapacity": 1, "maxCapacity": 10, "targetValue": 100, "scaleInCooldown": 600, "scaleOutCooldown": 120, }, }) SagemakerAutoscalingConstruct( self, "Autoscaling", props=SagemakerAutoscalingProps( endpoint=your_endpoint, # your CfnEndpoint autoscaling=AutoscalingConfig( min_capacity=1, max_capacity=10, target_value=100, scale_in_cooldown=600, scale_out_cooldown=120, ), config_hash=config_hash, ), ) How the construct works This section walks through the key implementation details of SagemakerAutoscalingConstruct . Deregister The Deregister custom resource calls deregister_scalable_target before the endpoint update. It handles ObjectNotFoundException for first deployments where no scalable target exists: def handler(event, context): resource_id = event["ResourceProperties"]["ResourceId"] if event["RequestType"] in ("Create", "Update"): try: client.deregister_scalable_target( ServiceNamespace="sagemaker", ResourceId=resource_id, ScalableDimension="sagemaker:variant:DesiredInstanceCount", ) except client.exceptions.ObjectNotFoundException: pass # No scalable target on first deploy return {"PhysicalResourceId": f"deregister-{resource_id}"} The dependency is established in CDK: props.endpoint.node.add_dependency(deregister) Register Scalable Target Even though CloudFormation treats the endpoint resource as complete once the update is accepted, the scalable target can only be registered after the endpoint reaches InService . Rather than block a single Lambda invocation on that wait, Register Scalable Target uses the provider’s asynchronous completion pattern. The on_event handler returns immediately, and the provider’s waiter re-invokes the is_complete handler on query_interval until the endpoint is InService . The provider’s total_timeout (1 hour) bounds the wait, not the timeout of any single Lambda invocation: def handler(event, context): props = event["ResourceProperties"] resource_id = props["ResourceId"] endpoint_name = resource_id.split("/")[1] if event["RequestType"] == "Delete": return {"IsComplete": True} status = sm_client.describe_endpoint(EndpointName=endpoint_name)["EndpointStatus"] if status == "Failed": raise Exception("Endpoint in Failed state") if status != "InService": return {"IsComplete": False} as_client.register_scalable_target
With Amazon Quick Sight layer maps, you can now plot points and routes, like store locations across a city or flights leaving an airport. Geospatial maps help businesses spot patterns that are tied to location, like how a region is performing or where customers are concentrated. Amazon Quick Sight supports this with multiple map types, including point maps, filled maps, and layer maps. When layer maps first launched , you could upload a GeoJSON file to draw custom shape layers, such as sales territories or districts, and join those shapes to a dataset. That handled regions and boundaries well, but it didn’t cover the individual locations inside them or the routes running between them. Layer maps now support two more geometry types from the same GeoJSON workflow: Point and LineString . Points mark specific locations, such as individual store or facility sites. A LineString traces the route between locations, like a delivery path or a flight between airports. Amazon Quick Sight , the business intelligence (BI) capability of Amazon Quick , is a unified BI service. It delivers modern interactive dashboards, natural language querying, pixel-perfect reports, machine learning (ML) insights, and embedded analytics at scale. Amazon Quick brings together AI agents for business insights, research, and automation in one integrated experience, helping you work smarter and faster while maintaining security and access policies. In this post, we walk through the dataset and GeoJSON structure that layer maps require. We also explain how points and lines work in a layer map visual and when to choose each. Finally, we cover advanced techniques for controlling the viewport, layering geometries, and updating GeoJSON files programmatically. Dataset structure considerations A layer map gets its shapes from a GeoJSON file and its metrics from a Quick Sight dataset, and a shared key joins the two. How you structure each one determines what the map can show. This section covers what you need before you start and how a GeoJSON file is organized. Prerequisites The following are the prerequisites that you need before creating a layer map in Quick Sight: A GeoJSON file describing your geometry. A Quick Sight dataset holding the metrics and dimensions. A join key shared between the Quick Sight dataset and the GeoJSON file. The GeoJSON file must be saved in .geojson format, with a maximum supported file size of 100 MB. Structure of the GeoJSON file GeoJSON is a JSON-based format for describing geographic features such as points and lines. { "type": "FeatureCollection", "features": [{ "type": "Feature", "geometry": { "type": "Point", "coordinates": [88.4, 22.6] }, "properties": { "prop0": "value0" } }, { "type": "Feature", "geometry": { "type": "LineString", "coordinates": [ [88.2, 22.4], [88.9, 23.1], [89.5, 22.8], [90.1, 23.6] ] }, "properties": { "prop0": "value0" } }] } The structure of the file is as follows: FeatureCollection is the whole file. It has a type of FeatureCollection and a features array. Feature is a single map element. It carries a geometry (the shape) and a properties object, which holds the join key. geometry has a type (Point, LineString, or Polygon) and its coordinates . The coordinates hold a single [longitude, latitude] position for a Point , and an ordered list of positions for a LineString . Configuring the Quick Sight dataset and its GeoJSON In Quick Sight, when you configure a layer in a layer map visual, there are two primary settings to consider. The first is the Geometry type to visualize , which sets which geometry from the GeoJSON file renders: Points, Lines, or Polygons. A layer renders one geometry type at a time, so when a file mixes types like the preceding example, only Features of the selected type appear on the map. The second is matching the Shape file key field to the Dataset key field , which links each Feature in the GeoJSON to a row in your Quick Sight dataset. The field names don’t need to be identical, but the values do. The following two examples show this end to end, first a LineString layer and then a Point layer. Example: LineString layer Consider example data showing a flight from JFK to a destination city like Los Angeles. The GeoJSON structure for a single datapoint is configured as follows: { "type": "Feature", "geometry": { "type": "LineString", "coordinates": [ [-73.7781, 40.6413], [-85.5627, 40.6407], [-97.1415, 39.4569], [-108.1612, 37.1753], [-118.4085, 33.9416] ] }, "properties": { "route_id": "JFK-LAX" } } The corresponding row in the Quick Sight dataset contains the following information: Figure 1: Dataset row for the JFK-LAX route, keyed by route_id The route_id field is used to join the Quick Sight dataset with the GeoJSON file. The GeoJSON holds one such feature per route, and the dataset one row per route. The Geometry type to visualize is set to Lines. Figure 2: The JFK-LAX route drawn as a LineString layer Example: Point layer Consider example data showing a retail store location like Long Island City. The GeoJSON structure for a single datapoint is configured as follows: { "type": "Feature", "geometry": { "type": "Point", "coordinates": [-73.9490, 40.7447] }, "properties": { "store_id": "S-109" } } The corresponding row in the Quick Sight dataset contains the following information: Figure 3: Dataset row for a store, keyed by store_id The store_id field is used to join the Quick Sight dataset with the GeoJSON file. The GeoJSON holds one such feature per store, and the dataset one row per store. The Geometry type to visualize is set to Points. Figure 4: A store plotted as a Point layer Lines on layer map visuals Layer map visuals now support LineString geometry, connecting an ordered list of coordinates into a single path. As covered in the data structure section, each route is one feature, and its coordinates are the ordered positions the map connects in sequence. Use lines when the relationship between locations is what you want to show, such as flight paths, delivery routes, shipping lanes, or pipeline segments. The visual draws each path from the coordinates in your GeoJSON file rather than generating it from your data at load time. A field from your joined dataset can determine the line color encoding, so you can group routes by whatever that field represents. In addition to color, you can customize lines with width and opacity controls to adjust visual prominence and clarity on the map. The join key in the GeoJSON properties object ties each LineString to the dataset row behind its tooltip and color. For example, an airport operations team might map the flights leaving New York (JFK) on a given day. The GeoJSON file holds one LineString per route, keyed by a route ID that joins to the dataset. Each line starts at JFK and ends at its destination, such as Los Angeles, Chicago, or Miami. The dataset supplies the origin, destination, and carrier that show up in the tooltip. Figure 5: Flights leaving JFK drawn as LineString routes to multiple destinations Points on layer map visuals Amazon Quick Sight offers a native point map visual type that geocodes location names automatically and refreshes with your SPICE schedule or Direct Query connection. It supports dynamic point sizing, color encoding, and point styles such as cluster points and heatmap density gradients. It works best with data that changes frequently and where you want the map to respond to those changes visually. With the added support for GeoJSON point layers in layer map visuals, Quick Sight now offers a second point mapping capability with a few key differences. Layer map visual points use a uniform marker size and don’t support geocoding, so you provide explicit coordinates in the GeoJSON file. In return, you can control the viewport precisely, locking the map to an exact geographic region, and disabling panning and zooming. The following table summarizes the key differences: Feature Point map Layer map visual points Data source Latitude/longitude values or geolocation names in your dataset GeoJSON Point features joined to your dataset by a shared key Geocoding Recognizes city, county, state, country, and postal code automatically Not available. Must provide explicit coordinates in the GeoJSON file Location updates Mapping updates automatically with SPICE refreshes or Direct Query as location values change Visual uses a static GeoJSON file for point locations. Requires manual re-upload or API call to update geometry Viewport Auto-fits to your data, panning and zooming always enabled Auto-fits by default, can configure if panning and zooming is allowed Appearance Size and color can be encoded from the dataset. Point style configuration available (basic, cluster, heatmap) Customizable uniform marker size, fill opacity, border color and thickness. Color can be encoded from the dataset Choose layer map visual points when: You require a locked, non-scrollable viewport. Your locations are fixed and curated (such as regional offices, distribution centers, retail stores) rather than changing with each data refresh. You plan to combine points with other geometry types (covered in the advanced use cases section). Consider a retail brand whose regional managers use a Quick Sight dashboard to track store performance across a metro area. The GeoJSON file contains a Point feature for each store with a store ID property that joins to the dataset. The dataset holds the metrics (store type, monthly revenue, daily foot traffic) that drive the tooltip and color encoding. You position the map at the zoom level covering the metro area, then disable map navigation so the map renders at a consistent view on every load. Managers can’t accidentally zoom out to a national or global view. Figure 6: Store locations as points with map navigation disabled to lock the metro-area view Advanced use cases The previous sections cover a standard layer map setup. Use the following techniques when you need more control over what readers see, want to combine geometry types in a single view, or need geometry that changes over time. Controlling the viewport When you need to control the viewport in layer map visuals, you can pan and zoom to the view you want manually. Then disable map navigation to lock it in place. For use cases where you need more precise boundaries or programmatic control, you can set the exact viewport coordinates through the visual definition using the Quick Sight APIs. The layer map visual stores its viewport configuration in the MapState property within ChartConfiguration . This property accepts two fields: Bounds : The geographic coordinates defining the minimum visible area (North, South, East, West). MapNavigation : ENABLED or DISABLED to control whether readers can pan and zoom. Although the bounds property is not exposed on the console, you can set these bounds when working with the dashboard definition JSON for programmatic asset management. For example, to define the bounds, modify the MapState block in the visual definition, then call the CreateDashboard or UpdateDashboard APIs. The bounds define the minimum area that is guaranteed to be visible. The map zooms to make sure this entire area fits within the visual container. For the tightest fit with minimal extra space added to the map, match the aspect ratio of your bounds to the shape of your visual container. Layering geometries For use cases that require combining multiple geometry types into a single view, you can layer visuals on top of each other. For example, the following screenshot demonstrates plotting customer locations as points within US regional boundaries defined by polygons. This composition shows which territories have the highest customer density and can be used to correlate with other metrics present in the dataset like performance. Figure 7: Customer points layered within polygon regional boundaries This second example demonstrates combining points as stops along a delivery route path. The LineString draws
Plant breeding decisions depend on data that spans genomics, field trials, imagery, environmental conditions, and model outputs. Bayer Crop Science runs hundreds of machine learning (ML) model configurations against this data across crops, regions, and teams, and each run needs to be repeatable and governed. To meet that requirement, Bayer built the Connected Decision Engine (CDE), a system for deploying, orchestrating, monitoring, and reusing ML workflows. CDE gives breeders, analysts, researchers, data scientists, and data engineers a consistent way to run analytics while preserving execution history and operational controls. As adoption grew, the bottleneck moved from running workflows to understanding them. Explaining a result meant piecing together context from APIs, documentation, metadata stores, dashboards, and logs. A user who wanted to know why a workflow failed, what a model did, or where a result came from still had to know which system to inspect. To close that gap, Bayer built the Agentic Decision Engine (ADE), a multi-agent, natural language interface over CDE. A supervisor agent routes each question to a specialist agent, the specialist retrieves context from the relevant system, and the user receives an answer without navigating the operational stack. In its first phase, ADE resolves about 80 percent of user questions without an engineer in the loop. ADE runs on Amazon Bedrock AgentCore , an AWS service to build, connect, and optimize agents at scale with any framework or model. AgentCore let the team focus on agent behavior instead of building and operating its own agent hosting, session isolation, and tool access layer. Three AgentCore capabilities drove that choice: a managed runtime that isolates each user session, AWS Identity and Access Management (IAM)-scoped tool access through AgentCore Gateway, a capability of Amazon Bedrock AgentCore, that keeps authentication and authorization outside agent code, and a framework-agnostic design that let the team bring its own LangGraph orchestration and choice of models. To prepare for launch and to operate ADE afterward, Bayer engaged AWS Unified Operations . The engagement provided a designated Technical Account Manager (TAM) and a generative AI Domain Specialist Engineer (DSE), who worked as an extension of the ADE team from design through production. The TAM, as Bayer’s strategic advisor, brought an established understanding of Bayer’s operating model and business priorities. The DSE brought generative AI domain expertise on AWS. In three technical standups each week, the team reviewed and made design decisions. The DSE built working knowledge of the ADE workload and how the Bayer team operates, so technical guidance reflected both AWS service behavior and Bayer’s context. This post follows that journey: the ADE architecture, five design decisions made during the build, the work to prepare the platform for production, and how it is operated after launch. Architecture overview ADE is organized into four layers: the user experience, agent orchestration, specialist agents, and the tools and data sources those agents call (Figure 1). Users reach ADE through a React web front end, and requests pass through an identity proxy for Microsoft Entra ID before they reach the agents. GitHub Actions builds the agent container images, pushes them to Amazon Elastic Container Registry (Amazon ECR) , and deploys them to AgentCore runtime, a capability of Amazon Bedrock AgentCore. Amazon Bedrock AgentCore runtime hosts the agents, and because the runtime is framework-agnostic, the team runs its own LangGraph-based orchestration on it directly. The team chose LangGraph for its explicit graph-based control flow and checkpointing for multi-step workflows. LangGraph orchestrates the workflow, with a Supervisor Agent as the entry point that classifies each request, selects a specialist, and combines results when a question spans domains. The data plane is separated by responsibility. Amazon DynamoDB stores LangGraph checkpoint state. Amazon Aurora PostgreSQL stores session records, user feedback, and analytics. Amazon ElastiCache caches agent responses with a time to live (TTL). Domain content in Amazon Simple Storage Service (Amazon S3) is embedded with Amazon Titan Text Embeddings V2 and indexed in Amazon OpenSearch Serverless for Retrieval Augmented Generation (RAG). Amazon CloudWatch captures runtime and application telemetry. AgentCore Gateway exposes backend APIs to agents as Model Context Protocol (MCP) tools and keeps authentication and authorization outside agent code. Figure 1: Agentic Decision Engine architecture on Amazon Bedrock AgentCore ADE runs 22 specialist agents in production, each scoped to one domain with its own tools and data sources. For example, the Domain Expert Agent answers conceptual questions through RAG. The Operational Agent retrieves live workflow status and results from CDE. The Troubleshooting Agent analyzes operational telemetry. The Visualization Agent generates charts or returns existing dashboards. A customer-specific Models as Skill pattern converts model code and documentation into skill definitions stored in a central registry, so agents answer model-specific questions from the model’s implementation instead of generic model knowledge. When a breeder asks, “Why did my phenotyping workflow fail yesterday?”, the supervisor classifies the intent and routes the request to the Troubleshooting Agent. That agent reads the workflow’s telemetry in CloudWatch and its status from the CDE API through AgentCore Gateway, identifies the failure point, and returns a diagnosis through the supervisor (Figure 2). The breeder doesn’t search logs directly. With the Troubleshooting Agent, a diagnosis that once took over an hour of manual log-hunting and waiting for engineer support now takes under 10 minutes. Figure 2: ADE request flow: intent classification, cache lookup, specialist routing, and response Five design decisions on the path to production The Unified Operations TAM and DSE took part in the following decisions as members of the working team, not as an escalation point after the fact. That let Bayer test each design against AWS service behavior before committing to it, while keeping ownership of the architecture and the final call. 1. Externalize orchestration state without treating the runtime as stateless AgentCore runtime provides isolated sessions that preserve execution context across invocations within a session’s lifecycle. That context isn’t a durable store for a multi-step workflow. ADE writes LangGraph checkpoints to DynamoDB so the graph can resume after scale-out events, restarts, or session boundaries without depending on one execution environment remaining available. Principle: Use the runtime session for active execution, and persist the state required to resume or reconstruct the workflow in a durable service. 2. Separate enterprise identity, gateway access, and downstream authorization Bayer standardizes on Microsoft Entra ID for user identity, while ADE’s services authorize each other with AWS-native mechanisms. Rather than force one token across every trust boundary, the team gave each boundary its own mechanism. Entra ID remains the source of user identity. AWS Identity and Access Management (IAM) authorizes service-to-service calls. AgentCore Gateway outbound authorization handles access to downstream CDE services. A request interceptor AWS Lambda function performs custom validation and passes the authorization attributes that downstream tools require. The team worked through the token contract at each hop with the DSE. Principle: Identity failures in agent systems cluster at the transitions between user identity, agent identity, and tool identity. Define the issuer, audience, claims, scopes, and credential lifetime at every hop instead of reusing a token that is valid for one boundary at the next. 3. Treat model selection as an interface contract During the build, the team evaluated a newer model version with reasoning mode enabled. For that version, the combination of tool use and structured output that ADE depends on was not compatible with how its agents were implemented. The DSE confirmed the constraint, and the team retained the current model for the release. Principle: For tool-using agents, model choice is constrained by the full interface: tool calling, structured output, streaming behavior, reasoning features, latency, quota availability, and model-specific request semantics. A newer model is not automatically a drop-in replacement. 4. Control the tool surface before it controls the prompt CDE manages more than 500 model configurations. A dedicated MCP server per API would have added deployment, authentication, and version management overhead. ADE instead registers OpenAPI specifications with AgentCore Gateway, which exposes the REST operations as MCP tools. Adding or changing an API becomes a schema and gateway configuration change instead of a new server deployment. Tool schemas and descriptions also consume input tokens and compete with the user’s request and retrieved context. ADE scopes tools by specialist agent instead of giving every agent the full catalog. For larger catalogs, AgentCore Gateway supports semantic tool search, so an agent can find relevant tools without loading the entire catalog on every turn. Principle: Scope the tool catalog per agent. Every tool definition adds tokens and widens the model’s decision space. 5. Promote complex capabilities from tools to agents Visualization started as a tool invoked by other agents. As it grew, it needed its own data retrieval, chart selection logic, retries, error handling, and a decision about whether to generate a chart or return an existing dashboard. The team moved it into a specialist Visualization Agent. The agent generates on-demand charts with matplotlib and Plotly in AgentCore Code Interpreter, a capability of Amazon Bedrock AgentCore that provides a sandboxed code execution environment, and exchanges data with it through Amazon S3. When a prebuilt dashboard is the better answer, the agent returns the existing Amazon Quick or Power BI asset. Principle: A tool performs a bounded operation. When a capability starts to own state, retries, planning, policy, or presentation decisions, model it as an agent. Preparing ADE for production As the build moved toward launch, the focus shifted from design to operability: detecting agent failures, attributing inference cost, and hardening the deployment. Observability: Measure task failures, not only exceptions A supervisor architecture turns one user request into a chain of model calls, tool calls, and agent handoffs. The final answer can be wrong even when every HTTP request succeeded. ADE encountered this directly: application exception handlers caught agent errors and returned responses that monitoring counted as successful. After reproducing the issue with the DSE, the team added custom CloudWatch metrics inside failure paths and alarms on agent-specific conditions: tool call errors, throttling, model timeouts, and structured output validation failures. Traces and invocation logs follow each request across the supervisor and specialists. Principle: Distinguish transport success from task success. Agent observability should show whether the task completed, which agent or tool changed the outcome, and whether the response used the expected source. Cost attribution: Combine billing attribution with request-level telemetry One ADE request can invoke several models and tools, so account-level Amazon Bedrock spend doesn’t explain unit economics. The team designed the attribution approach with the DSE using three mechanisms that answer different questions. Amazon Bedrock application inference profiles attribute billed model usage to an application or workload. Request metadata and model invocation logging associate token usage with a specific agent, feature, or re
Real-world 3D reconstruction pipelines often rely on multi-view images degraded by lossy compression. While compression artifacts have been extensively studied in single-image or video restoration, their impact on multi-view geometry remains largely overlooked. In this work, we identify compression-induced cross-view geometric inconsistency as a critical failure mode, where view-dependent artifacts disrupt feature matching, dense correspondence, camera parameter estimation, and downstream 3D reconstruction tasks such as novel view synthesis. To address this problem, we propose CompMVR, a compression-aware multi-view diffusion restoration framework that jointly refines compressed views while preserving cross-view consistency, and construct compressed datasets for systematic evaluation. Unlike conventional restoration methods that optimize each image independently for 2D perceptual quality, CompMVR leverages learned compression priors and cross-view correspondence to recover visual details while improving geometric reliability across views. Experiments across diverse datasets, codecs, and compression levels demonstrate gains in restoration quality and downstream 3D tasks, including novel-view synthesis, camera pose estimation, and view matching. These results establish compressed multi-view restoration as a distinct problem and highlight its importance for 3D reconstruction under lossy compression. Code and data for this paper are available at https://mchaewon.github.io/CompMVR-project-page/.
中文 | English 体验优化 新增左侧栏、会话视图和右侧栏的「更多」菜单，方便浏览和创建插件。 @Yifffan , @imccyu 修复切换会话后，点击「新会话」找不回未发送草稿的问题。 @imccyu 其他变更 DeepSeek API Key 适配器的模型列表更新方式与账号登录模式保持一致。 @LegGasai Improvements Add More menus to the left sidebar, conversation views, and right sidebar for browsing and creating plugins. by @Yifffan , @imccyu Fix New Session losing an unsent draft after switching sessions. by @imccyu Chores The DeepSeek API key adapter now updates its model list the same way as account sign-in. by @LegGasai Full Changelog: dsh-v0.2.1-rc.2...dsh-v0.2.1-rc.3 官方 Release：dsh-v0.2.1-rc.3。请查看 Release notes、依赖和许可证变化。
Teaching a robot to navigate an obstacle course requires more than a walking policy. It must perceive its surroundings, maintain balance, and adjust its actions... Teaching a robot to navigate an obstacle course requires more than a walking policy. It must perceive its surroundings, maintain balance, and adjust its actions when obstacles block the path ahead. Developing these capabilities connects several stages: creating assets and an environment, using physics simulation to model how the robot interacts with its environment, and training and evaluating… Source
Unrecognized anomalies in critical infrastructure can rapidly cascade into catastrophic physical or operational failures. Current anomaly detection approaches often rely on direct access to the monitored system by utilizing onboard sensors, such as motor encoders, to detect anomalous states. However, legacy hardware often lacks these sensors, or strict security protocols may prohibit access altogether. We therefore present AWARE, a world model framework that is capable of anomaly detection using only external camera monitoring via system dynamics estimation and state prediction. Evaluated across 240 real-world robotic crane trajectories, including payload collisions and motor degradations, AWARE demonstrates a 26% improvement in anomaly classification over our strongest baseline under noisy, camera-only state estimation. Furthermore, the inferred dynamics parameter can be decoded to identify the source of faults, localizing the degrading motor with 88% accuracy.
We introduce SemGeo-Gen, an unsupervised method for generating approximate cross-instance semantic-geometric supervision from weakly aligned 3D object collections. Our goal is not to recover exact point-level correspondences, which are often ambiguous across different instances and would require expensive, impractical manual supervision at scale. Instead, we automatically produce approximate correspondences that are semantically meaningful, geometrically consistent, and diverse enough to train modern correspondence models. Given only coarse category-level rotation alignment, SemGeo-Gen lifts multi-view DINOv2 features to 3D, aligns object instances using continuous piecewise-affine registration, and prunes candidate matches using semantic consistency. The resulting 3D correspondences can be projected into rendered views, yielding scalable 3D-3D, 3D-2D, and 2D-2D supervision without manual annotation. We validate the generated correspondences against sparse human annotations from KeypointNet, obtaining 93% PCK@0.10 and 97% PCK@0.15. More importantly, we show that using the generated correspondences for synthetic pretraining improves recent state-of-the-art semantic correspondence models on SPair-71k after fine-tuning on real data, with gains of up to 5 PCK@0.10 points. Together, these results indicate that approximate semantic-geometric supervision generated from 3D assets can improve real-image correspondence learning and serve as a scalable alternative to costly manual annotation. Project page: https://bgu-cs-vil.github.io/SemGeo-Gen/.
Resource-constrained robots and autonomous vehicles need scene memories that localize a camera and render nearby views without a desktop GPU. Scene-coordinate regression (SCR) and 3D Gaussian splatting (3DGS) fit because they avoid an SfM stage, but ACEsplat-style pipelines are GPU-centered, and moving them to an edge FPGA is hard because SCR is pose-sensitive and 3DGS rendering is irregular and capacity-sensitive. We present LENS, an edge scene-memory acceleration system on the AMD KR260 that brings an ACEsplat-style workload to a low-power FPGA/ARM platform. Rather than proposing a new scene model, LENS co-designs the two dominant kernels with the board data path: an INT16 SCR accelerator that preserves pose accuracy, and a bit-exact fixed-point ordered 3DGS renderer that rewrites alpha transmittance as a log-sum-exp recurrence and removes provably useless streamed Gaussians. All LENS latency, power, and quality results are board-measured. The SCR engine preserves DSAC* localization accuracy at 25.42 ms, the on-board renderer reaches 34.11 dB on Cambridge against ACEsplat's 33.20, novel-view synthesis on RealEstate10K reaches 27.84 dB against 26.11, and all ten Wayspots scenes reach the ACEsplat rendering target. LENS wins performance per watt over a Jetson Orin NX on every deployment task, by 1.9x to 5.7x at about a quarter of the power, and is faster in absolute latency on localization and Wayspots rendering. On-device training reaches ACEsplat-level quality but not the GPU's time or energy, marking the remaining boundary for a future training accelerator.
Cinematic video generation requires both temporal inference from limited inputs (i.e., sparse rather than frame-by-frame guidance) and factor-aware control of character, scene, action, and camera. Multimodal large language models (MLLMs) offer useful priors for both capabilities, but their representations are not automatically actionable for video generation: a Video Diffusion Transformer (DiT) may bypass them through native conditioning pathways, and using them does not ensure either factor-aware control or generation-compatible temporal prediction. We propose Factor-Aware Predictive Learning (FAPL), which establishes effective MLLM latent use as a shared prerequisite and then targets the two capabilities with complementary learning signals. First, latent-only generator grounding trains the DiT with a frozen MLLM and connector, strengthening the DiT's use of MLLM latents for generation. Second, for factor-aware control, paired videos specify a target-factor change and non-target preservation, and factor-span replacement trains the DiT to realize this relationship in its output. Third, for temporal inference, text-only and text-plus-complete-video observations share a video target, and the trained DiT is then frozen while the MLLM and connector are optimized through it. This final stage trains limited-input representations for generation while retaining complete-observation reconstruction as a complementary training mode. Extensive experiments demonstrate our consistently better performance on three cinematic video generation tasks (i.e., sparse-to-video, factor-aware editing, and multi-shot generation) over baselines and validate FAPL's contributions to MLLM feature use, temporal control, and factor coordination.
Pre-consultation medical dialogues are essential for collecting comprehensive patient information, but are costly to obtain at scale due to privacy and ethical constraints. Existing LLM-based synthesis methods often rely on prompt engineering or rigid rules, limiting their ability to balance clinical relevance with realistic patient diversity. We propose CoRPS-Dial, a controllable dual-agent framework for pre-consultation medical dialogue synthesis that integrates patient personas with guideline-aligned clinical workflow rules. CoRPS-Dial generates dialogues through interactions between a persona-driven patient agent and a process-guided doctor agent, followed by post-hoc quality control for role consistency, workflow completeness, and textual consistency. Experiments on coronary heart disease demonstrate that CoRPS-Dial improves alignment between synthetic and real dialogues in information coverage, clinical compliance, and dialogue naturalness. Downstream fine-tuning experiments further confirm the utility of the generated synthetic data for medical dialogue tasks. These results highlight the effectiveness of CoRPS-Dial for clinically grounded and controllable medical dialogue synthesis.
OpenAI explains how we strengthened containment for frontier model training and evaluation, including offline environments and high-compute red-teaming.
Continuous visual world simulation requires coupling evolving scene geometry with temporally persistent appearance. We present UniWorld, a unified framework that integrates depth trajectories and reference appearance within a shared video generation model. The central design maintains visual and geometric histories on a common timeline. Noise-aligned overlapping continuation places generated history at the prescribed sampling noise level and commits only newly generated latents. Rolling depth latent state preserves the corresponding geometric controls, incrementally reads newly available geometry, and keeps the conditioning fixed within each sampling window. Together, these mechanisms ensure exact preservation of latent history while bounding the active depth state by the window size. We evaluate continuous generation, paired history interventions, and long-horizon behavior across diverse scene categories. The same generator supports estimated, native, and on-demand simulator depth, while cached and incremental execution paths produce bitwise-identical outputs. Appearance changes and geometric control branches operate through the same interface without modifying committed history. The measured steady-state DiT execution path achieves real-time generation throughput, demonstrating the efficiency of UniWorld for continuous visual world simulation.
Co-speech motion generation remains challenging over long horizons due to the need for temporal coherence and consistent style. Existing methods typically generate long motions segment by segment, either conditioning on a short seed of recent frames, which gradually drifts, or retaining ever-longer explicit histories, which are costly to maintain and hard to learn from limited data. We trace these limitations to a key asymmetry between past and present: past motion carries slowly varying style that should be compactly summarized, whereas the current segment exhibits fast-changing local dynamics that require fine-grained modeling. This asymmetry motivates a simple Past-As-State, Present-as-Attention principle for long-horizon co-speech motion generation. Following this principle, we propose PASPA, which summarizes past history with an inter-block persistent state and models the current segment with intra-block bidirectional self-attention. We instantiate PASPA within a blockwise flow matching framework, with multi-block supervision for effective training over variable histories, prefill-decode state caching for efficient rollout, and hybrid classifier-free guidance for enhanced history and speech conditioning. Extensive public-benchmark experiments show that PASPA achieves state-of-the-art FGD performance and preserves motion style across long-sequence generation.
Feed-forward 3D Gaussian Splatting (FF3DGS) reconstructs 3D scenes from images in a single forward pass, making it well-suited to applications that must operate within limited resource budgets. To further improve its practicality, recent budget-controllable FF3DGS methods allow the number of Gaussians to be adjusted at inference, so that the output meets a given storage limit or rendering-latency constraint. However, these methods overlook the cost of inference itself: to produce a representation with the target number of Gaussians, they first predict a dense set of Gaussians and then prune or select from it, requiring nearly constant latency and peak memory regardless of the budget. We therefore propose RLSplat, a resource-linear budget-controllable FF3DGS framework whose representation size, inference latency, and peak memory all decrease linearly as the Gaussian budget is reduced. Instead of pruning or selection, RLSplat learns a Gaussian allocation policy that distributes the budget across local scene regions, so that exactly the requested number of Gaussians is generated. These Gaussians are then refined through resource-linear Gaussian decoding, in which each Gaussian attends only to nearby Gaussians found via an efficient hierarchical neighbor search, so that the decoding cost decreases linearly with fewer Gaussians. In zero-shot evaluation, RLSplat outperforms prior budget-controllable methods at every budget, surpassing the strongest baseline even with 4$\times$ fewer Gaussians, while running up to 4.1$\times$ faster with 9.6$\times$ less peak memory at low budgets.
Single-image 3D scene construction is a fundamental problem in 3D vision. Recent methods such as SAM-3Dfy enable end-to-end reconstruction of 3D assets and their scene layouts, but still struggle with fine-grained scene geometry and asset quality. Meanwhile, specialized 3D object generators provide detailed object geometry, while 3D foundation models offer strong and generalizable spatial understanding. We propose Easy-3Dfy, a compositional workflow that decouples object generation from scene layout estimation, allowing specialized models to fully exploit their respective strengths. An RPF-based aligner then bridges these capabilities by establishing geometric correspondence between object-local meshes and partial scene pointmaps, recovering object layout for scene composition. Without requiring complete scene-level supervision, the proposed workflow consistently outperforms SAM-3Dfy across multiple geometric metrics on Scene-Maker and 3D-FUTURE200. Our results demonstrate that explicit geometric composition can effectively combine high-quality object generation with strong spatial understanding, while enabling the workflow to benefit independently from advances in both 3D object generation and foundation models.
Modern autonomous driving systems widely adopt multi-view Bird's-Eye-View (BEV) based 3D perception models due to their superior performance. Despite their success, the robustness of these models against adversarial camouflage attacks remains largely unexplored. Existing camouflage attacks primarily target single-view 2D detectors, limiting their effectiveness in multi-view 3D perception settings. To address this gap, we propose BEVCA, a novel adversarial camouflage generation framework tailored for multi-view BEV-based 3D perception. Our framework integrates a BEV-feature-based adversarial loss with a multi-view neural rendering module, enabling effective and transferable camouflage attacks across different models and tasks. Extensive experiments in both digital and physical settings demonstrate that BEVCA outperforms state-of-the-art baselines and exhibits strong robustness under real-world conditions. Our code is available at https://github.com/WAVE-AT-IDEA/BEVCA.
Reinforcement learning has gained attention for improving action-conditioned world models (ACWMs), as it enables reward-driven optimization of visual quality (VQ) and action-following fidelity (AF) beyond flow-matching optimization. Realizing this potential requires reward signals that reliably rank videos by both VQ and AF. However, existing rewards capture only one dimension: metric-based rewards assess VQ but require ground-truth videos and do not directly assess AF, while inverse dynamics model (IDM) rewards assess AF without references but do not guarantee VQ. Moreover, IDMs, typically trained on real videos, may provide unreliable rewards for generated videos containing visual artifacts and physical inconsistencies. In this work, we introduce the Action and Video Mismatch Benchmark (AVM-Eval), comprising 67.2k pairs that evaluate how reliably IDMs assess AF on real and generated videos. We then propose ACWM-RM, a reward model that jointly assesses VQ and AF without ground-truth videos during reward computation. To train it, we first construct the Action and Video Mismatch Preference Dataset (AVM-Pref) with 506.3k pairs providing complementary supervision: action-mismatch pairs supervise AF, while video-mismatch pairs supervise VQ and expose the model to generated videos. We then pretrain an action-conditioned predictor on real videos to learn forward dynamics before jointly training it with a patch-weighted score head on AVM-Pref to transfer these dynamics to AF assessment on generated videos. Evaluation on AVM-Eval reveals the limited reliability of IDMs on generated videos, while downstream RL experiments show that ACWM-RM outperforms reward baselines in both criteria.
Humans reason about the physical world by flexibly combining different forms of inference as a problem unfolds. Achieving this flexibility is a major open challenge in AI: capabilities often remain separated across specialized models, limiting how they can be composed and jointly adapted to a task. We introduce $Ψ$-0.6, a unified vision-language world model that adaptively composes perception, language, geometry, and simulation within a shared reasoning process we call streams of thought. These streams interleave images, text, and explicit representations of geometry and motion, all of which the model can generate and interpret through a shared token interface. $Ψ$-0.6 surpasses leading open-weight models zero-shot on multi-view spatial reasoning, with further gains from reasoning through geometric intermediates. It also supports counterfactual simulation: generating and interpreting imagined scenes to determine, for example, whether moving an object would obstruct a pathway. Reinforcement learning from task feedback jointly optimizes the selection, generation, and use of intermediate representations, improving visual question-answering accuracy with gains that transfer to unseen benchmarks. $Ψ$-0.6 offers a path toward adaptive physical reasoning -- not only predicting the physical world but also choosing the representations through which to reason.
### Added - Export any project as a ZIP with its settings, cover, and workspace, and import it as a new project on another computer. Interactive Story exports include referenced Library assets, and older Story project ZIPs remain importable. ([#355](https://github.com/WhiteTowerAI/ohmygame/pull/355) by @xuhongbo) - Drop, paste, or upload text documents, CSV/TSV tables, and more image, audio, video, and 3D formats into the canvas. Keep other files, including source, fonts, and ZIPs, as project file nodes with a desktop file-location action. ([#380](https://github.com/WhiteTowerAI/ohmygame/pull/380) by @dogxii, [#381](https://github.com/WhiteTowerAI/ohmygame/pull/381) by @dogxii) - Confirm folder imports before copying files, with choices to preserve the directory structure as one folder node, import files separately, or cancel. Repeated file and folder names are kept without overwriting existing imports. ([#382](https://github.com/WhiteTowerAI/ohmygame/pull/382) by @dogxii) ### Changed - Claude Code, Codex, and OpenCode now load your existing CLI settings, Hooks, MCP servers, Skills, and instructions while retaining OhMyGame's selected permissions. OpenCode requires version 2.1.0 or newer in the 2.x series; upgrade v1 before using it. If OpenCode reports saved project grants, remove those grants in OpenCode and check again. ([#376](https://github.com/WhiteTowerAI/ohmygame/pull/376) by @dogxii) - Community view counts now measure successful game-entry requests, including repeat visits, refreshes, and direct shared links, rather than unique visitors. Counts start at zero with the matching Cloud migration and survive updates to the same published game. Hosted deployments need the matching Cloud API and play-count migration. ([#353](https://github.com/WhiteTowerAI/ohmygame/pull/353) by @dogxii) ### Fixed - Keep Library and Community controls, plugin cards, and external-agent actions usable in narrow windows, including long plugin names and translated labels. ([#372](https://github.com/WhiteTowerAI/ohmygame/pull/372) by @dogxii)
Large language model agents that use tools could take on part of the electronic health record and desk work that occupies physicians' time, but in many hospitals the medical information systems run inside closed networks, so the model and the agent harness must also sit inside. We designed, implemented, and deployed LLENS, a clinical support AI agent platform that runs inside the closed network of Hokkaido University Hospital on one host with 8 NVIDIA H200 GPUs, the open-weight model Kimi K2.6 (~1 trillion parameters), and a web application rebuilt from Open WebUI. The harness has 10 components, including page image reading, full-text attachment input, a path-addressed document tree without a vector index, a network-isolated sandbox, artifact return, self-checking of generated output, subagents, and saved procedures. Kimi K2.6 was chosen from 3 open-weight candidates and scored 480/500 on the 2026 Japanese Medical Licensing Examination; it generated 119 tokens/s for one user and served 16 concurrent requests at 908 tokens/s in total. On 180 questions over hospital documents (3 seeds each), mean accuracy was 10.7% with no documents, 81.3% with fixed retrieval (top 10 chunks of one lexical search), and 97.2% with interactive access, in which the model reads with document tools and the sandbox (15.9 percentage points above fixed retrieval, 95% CI 10.9-21.3); the difference lay in filtering/counting questions (16.7% vs 87.8%). Five scenarios with synthetic data ran on combinations of the components: needlestick injury response, batch adverse event grading, scanned referral letter transcription, falls data aggregation and plotting, and investigation of 3 topics by subagents. The platform has run on the hospital network since September 14, 2026.
Video provides a natural source of supervision: what happens next serves as a learning signal for visual world modeling. Yet masked video pretraining focuses on recovering missing content from surrounding context rather than predicting the future from the past. Autoregressive methods instead predict forward one token at a time, but have yet to match joint-embedding methods in representation quality. This gap motivates us to revisit the granularity and space of future prediction. In this work, we introduce VANE, a self-supervised visual pretraining framework built on next-frame prediction in embedding space. A causal encoder summarizes past observations for a state predictor that infers the representation of the entire next frame, making temporal evolution the primary learning signal. This prediction is carried out in a fixed embedding space provided by a frozen semantic tokenizer, emphasizing scene content and structure. To retain fine-grained spatial detail alongside semantic content, a lightweight auxiliary generative decoder reconstructs future-frame patches conditioned on the predicted representations. To unify visual modeling across images and videos, we construct a local-to-global crop sequence for still images, allowing the same encoder to predict newly revealed regions from earlier crops under the same objective. Experiments show that VANE improves with model and data scaling, achieving SOTA on most anticipation, recognition, and dense perception benchmarks among unified encoders. Especially on EPIC-KITCHENS-100 action anticipation, it surpasses V-JEPA 2.1 by 7.1 points in mean-class recall@5. These results demonstrate that next-frame prediction in embedding space is effective and scalable for visual world modeling.
Feed-forward neural renderers synthesize global illumination across novel 3D scenes without per-scene optimization or runtime ray tracing. Existing transformer-based renderers tokenize individual triangles, so self-attention scales quadratically with mesh resolution. Yet macroscopic radiative transport is governed by the underlying surface geometry and object arrangements rather than a particular mesh discretization; finer tessellation can thus inflate token counts without changing the underlying transport physics, motivating a discretization-agnostic and object-centric representation. We introduce RADIANT (RADiance-based Inter-Actor Neural Transformer), an object-centric architecture for feed-forward global illumination from point inputs. By representing each object with a compact budget of 32 surface-patch tokens, RADIANT decouples attention sequence length from polygon counts. In a single network, an inter-object scene transformer models 3D radiative exchange and a view decoder renders images at target camera poses. To capture light transport without materializing O(N^2) bias matrices, invariant form-factor attention embeds geometric distances and surface orientations directly into query-key dot products as SE(3)-invariant inner products weighted by learned area and material gains. For primary visibility without mesh connectivity, camera rays cross-attend scene tokens guided by depth-tested point splats in a shared spherical frame. Across a 1.03M-scene benchmark under matched model capacity, RADIANT reaches 25 dB tone-mapped PSNR in 5x fewer training steps than RenderFormer, and achieves 32.2 dB at 600k steps (+3.3 dB over RenderFormer and +7.6 dB over RenderFormer-V2). We further demonstrate the scalability of RADIANT, which completes a 512x512 forward pass of a 500-object scene in 0.83 s, with computational cost scaling with the number of objects rather than polygons.
Kazakhstan’s AI ecosystem is moving from infrastructure plans to deployment. The shift is visible across the full AI stack — from infrastructure to models to applications. The announcements below — coinciding with a visit to Astana by NVIDIA founder and CEO Jensen Huang — point to the nation’s increasing supply of accelerated computing, along with its growing demand for what comes next. On Friday, Oct. 23, NVIDIA founder and CEO Jensen Huang visited Astana, meeting with President Kassym-Jomart Tokayev at the Akorda Presidential Palace and joining Deputy Prime Minister Zhaslan Madiyev — Kazakhstan’s minister of AI and digital development — for a public fireside chat at the Alem.AI program. Read All NVIDIA in Europe Announcements Jensen Huang and Kazakhstan’s AI Minister Speak at Alem.AI In their fireside chat at Alem.AI, Kazakhstan’s international center for artificial intelligence, Madiyev and Huang discussed the country’s push to become an AI nation — building the energy and computing infrastructure to support homegrown AI and applying it across the economy. Madiyev said Kazakhstan wants AI that reflects its language and culture. “We want AI that understands and speaks Kazakh and captures cultural contexts and cultural specifics,” he said, describing the country’s work on Kazakh-language models. Huang said Kazakhstan could turn its energy resources into a foundation for AI production — and a new kind of export. “Instead of just exporting energy, in the future, you could transform energy into intelligence and export intelligence,” he said. He also argued that local data is central to that ambition: it embodies a country’s culture, history and values. “Your country’s data is your natural resource. It belongs to you.” The conversation also turned to physical AI. Madiyev pointed to Kazakhstan’s factories, mines, agriculture, transport and logistics as potential areas for development, asking how the country could become a hub for training models for robots and other physical systems. Huang called the idea “fantastic,” saying physical AI could transform industries from mining and mineral processing to agriculture. Huang said AI and robotics can help people do more, but ambition will shape how fully they seize that opportunity. He applied the idea to Kazakhstan, saying AI could help lift the country and its people. “If you have ambition, and I believe you do, AI will lift your nation,” Huang said. “AI will lift your people.” Huang also praised the founders of Higgsfield AI, the Kazakhstan-founded startup, and it was announced NVIDIA is making an investment into the startup during the event. “The first thing and the last thing is you invest in people, not companies,” he said, describing the founders as visionary and pointing to their effort to compete globally. Asked about AI security, Huang said the technology must be developed and deployed responsibly. He described NVIDIA’s OpenShell as a way to contain AI agents by limiting their access to files and tools and monitoring their activity. For young people entering a fast-changing field, Huang recommended learning how AI works — not just how to use it. He said aspiring AI programmers should understand the technology’s principles, strengths and weaknesses, combining technical knowledge with the ability to work effectively with AI. “I envy being 18 years old today… because the world is changing so rapidly, and there’s a complete reset because of technology.” The Infrastructure Layer Is Scaling The visits coincided with a signing ceremony marking Firebird’s investment agreement with the government of Kazakhstan, held at the presidential palace. Building on its first AI factory in Armenia, Firebird is expanding into Kazakhstan, where it has secured a U.S. export license for 45,000 NVIDIA Grace Blackwell GPUs. The first phase is under construction with Kazakhtelecom at Data Center Valley, targeting commercial launch in the third quarter of 2027. A planned second phase would bring Firebird’s total potential AI infrastructure capacity in Kazakhstan to approximately 500 megawatts. Firebird also plans to establish Firebird Labs Kazakhstan to support local AI startups, research and talent development through funding, access to compute and research partnerships. Firebird cofounder and CEO Razmig Hovaghimian attended the signing ceremony at the Akorda Presidential Palace alongside government officials. “What began as a seed of an idea 180 days ago has grown into a major investment in Kazakhstan’s AI future,” said Razmig Hovaghimian, co-Founder and CEO of Firebird. “President Tokayev’s vision gave us the confidence to think bigger about what we can build together. With Jensen Huang here today, that ambition feels more tangible than ever. We’re building the infrastructure for Kazakhstan’s talent and ideas to reach the world.” Kazakhtelecom , the nation’s largest telecommunications company, has already deployed 400 NVIDIA H200 and 480 NVIDIA B300 GPUs in Astana and Almaty. Next year, the company plans to deploy more than 1,000 GPUs in the Data Center Valley , along with more than 1,000 NVIDIA Vera Rubin GPUs in the Ekibastuz region, serving diverse workloads with infrastructure that enterprises, developers and institutions can use at scale. An Honorary Black Belt for Huang At the end of their meeting, President Tokayev presented Huang with the Order of Dostyk, First Class, for his contribution to the development of artificial intelligence in Kazakhstan. Dostyk means friendship and the Order honors peace, friendship and cooperation. Later that day, President of the International Judo Federation (IJF) Marius Vizer and Timur Kemell, IJF director of promotion and development and a member of the IJF Executive Committee, presented Huang with an honorary black belt in Judo. The belt, embroidered in gold stitching, was a special thanks for Huang’s support of Kazakhstan’s AI ambition and for leadership aligned with the spirit and rigor of the martial art. AI-Native Companies Scale With Compute And that development is accelerating. The nation’s AI-native companies are also building out their compute capacity — and the applications that run atop them. Higgsfield is expanding its AI infrastructure with NVIDIA Vera Rubin running on cloud service providers, giving the company additional capacity as models, products and customer demand for its generative media applications grow. inDrive and Pony.ai are also collaborating to explore robotaxi services in Kazakhstan, with Astana as the first deployment city. The companies will assess the feasibility of bringing Pony.ai’s seventh-generation robotaxis to the inDrive app, supported by NVIDIA AI infrastructure, built on the NVIDIA Hyperion sensor and compute platform with the NVIDIA Halos safety system. More local capacity gives developers and companies room to train, fine-tune and serve increasingly capable models. Successful applications, in turn, create demand for more compute. Kazakhstan’s AI opportunity isn’t defined by GPU counts alone. Connecting that infrastructure with developers, universities, startups and industries can turn accelerated computing into useful applications. That was at the core of the discussion the NVIDIA team had with hundreds of developers from across Kazakhstan during a reception later in the evening. The gathering in this self-proclaimed AI nation was inspiring because of the ideas shared, the clear drive to build and a collective ambition to share it with the world. Today’s announcements represent different layers of the AI stack in Kazakhstan— infrastructure deployment, access to compute and AI-native product development — where the AI foundation is reaching new scale. Watch the fireside chat between Jensen Huang and Deputy Prime Minister Zhaslan Madiyev, streamed live from Alem.AI in Astana . Watch NVIDIA founder and CEO Jensen Huang’s GTC Berlin keynote in replay . Read All NVIDIA in Europe Announcements
OpenAI and Poland’s NASK are partnering on AI safety, security, and technical research to strengthen safeguards and support a more resilient digital ecosystem.
Video encoder models are increasingly used as world models for physical AI, where a policy plans by rolling out the latent, so what physics the latent holds bounds what the agent can plan over. Moreover, existing interpretability work probes quantities belonging to a single object; the quantities that make physics relational, such as momentum transfer, restitution, and friction, are defined between bodies and have not been examined. We probe relational physics attributes in two frozen self-supervised video encoders (V-JEPA2 and VideoMAE) under two-body settings and test if they transfer across environments. Our early findings are mixed. Both encoders separate elastic from inelastic collisions reliably. V-JEPA2's own predictor is less likely to predict a collision that creates kinetic energy compared to one that loses it. Momentum-violation detection appears in only one of the two encoders, a kinetic-energy readout survives a change of environment for V-JEPA2 but not for VideoMAE, and neither model holds a friction coefficient that survives a change of geometry.
Text-based person anomaly search requires a model to retrieve individuals from a large image gallery using natural-language descriptions of both appearance and behavior. This task is challenging because model training relies on synthetic image-text pairs, whereas evaluation is performed on real surveillance images containing domain differences, near-duplicate frames, and many visually similar distractors. We present a multi-model retrieval and reranking framework designed to improve synthetic-to-real transfer. The framework first adapts Qwen3-VL-Embedding-8B to synthetic person descriptions using bidirectional contrastive learning, explicit negative sampling, false-negative masking, and domain randomization. It then combines the normalized scores of six complementary Qwen3-VL checkpoints with those of a frozen SigLIP2 model using four-view gallery encoding. The fused retrieval scores identify the strongest candidates, which are jointly evaluated with the query using Qwen3-VL-Reranker-8B. The cross-encoder and retrieval scores are subsequently blended to produce the final ranking. We also construct a distractor-matched validation protocol entirely from synthetic data to select checkpoint weights, fusion coefficients, candidate depths, and reranking parameters without accessing the official test images. The final system achieves 94.7815% mAP and 90.8493% R@1 under the official AI City Challenge 2026 Track 4 test setting, with a reported eighth-place finish on the public leaderboard.
Amazon Polly Generative engine now offers more natural and accurate speech output. We’re introducing an upgrade to the Amazon Polly Generative engine that delivers more expressive, more accurate speech output with low latency. The upgrade adds 14 new voices across 9 locales, including general availability of Generative voices in Hindi and Finnish for the first time. The upgrade is now generally available in production. Importantly, this is a drop-in upgrade for existing generative voices. Voice names and API calls are unchanged, so your existing applications continue to work as they do today. Amazon Polly is a fully managed AWS service that converts text into natural-sounding speech for contact centers, large language model (LLM)-based agentic applications, media and entertainment, e-learning, accessibility, and other applications that require reliable voice output at scale. Hear the difference You can use sample audio to evaluate the improvements directly. The more natural, authentic, and expressive voices. Hear how Tiffany welcomes you in a delightful tone. “Hi there! I’m Tiffany, one of Polly’s generative voices. I was built to sound natural, expressive, and a little bit delightful. What should we talk about?” Accurate speech generation for numbers, currency, addresses, and so on. Kenneth reads back an order confirmation, including the tracking number, date and time, currency and address accurately. “Thanks for calling. Your order, number 4471 9902 ships today to 85 Harbor View Drive, Apartment 4B, Portland, Oregon 97209. It should arrive on Thursday, October 22nd, between 9 AM and 1 PM. Your total was $1,284.50 and your confirmation code is K-7-Q-2-X-9.” Net new languages. Arjun and Kiara for Hindi, and Suvi for Finnish. Hindi and Finnish are generally available for the first time on the Amazon Polly Generative engine. Arjun : नमस्कार, आप सुन रहे हैं आज की मुख्य ख़बरें। देश के कई हिस्सों में मानसून की विदाई के साथ मौसम सुहाना हो गया है। (Hello, you’re listening to today’s top stories. As the monsoon retreats across much of the country, the weather has turned pleasant.) Kiara : नमस्ते! मेरा नाम कियारा है। आइए, आज मैं आपको एक छोटी सी कहानी सुनाती हूँ।(Hello! My name is Kiara. Come, let me tell you a little story today.) Suvi : Majakanvartija ei ollut nähnyt laivaa yhteentoista päivään. Sitten, vähän puolenyön jälkeen, horisontissa välähti yksi ainoa valo. Kerran. Kahdesti. Ja sitten, uskomatonta kyllä, se vilkutti takaisin hänen nimensä. (The lighthouse keeper hadn’t seen a ship in eleven days. Then, just after midnight, a single light blinked on the horizon. Once. Twice. And then, impossibly, it blinked back his name.) Different styles of speech delivery based on textual context. Powered by the new Amazon Polly Generative engine, the voices can reflect emotion and prosody even better based on the text input, without manual setup. Neutral : “Your package arrived early.” Uncertainty : “So, um… your package arrived early. I think?” Surprised : “Wait, what? Your package arrived early?” New voices, languages, and Region availability A larger catalog overall for Amazon Polly Generative engine – Totaling 57 generative voices across 11 languages and 25 locales. 14 new voices across 9 locales – US English (Kenneth, Nate), Australian English (Russell, Curtis), Indian English (Kiara, Arjun), German (Tina), US Spanish (Carlos), Brazilian Portuguese (Leo, Carolina), Korean (Hyunduk), Hindi (Kiara, Arjun) and Finnish (Suvi). Net new languages generally available in Generative engine – Hindi and Finnish. Region expansion to AWS GovCloud (US-West) – Totaling 11 AWS Regions. Built for production-ready speech workflows for enterprise The upgraded Amazon Polly Generative engine uses a standard Amazon Polly API call to build your speech applications. With Amazon Polly, you can continue to build with familiar AWS patterns: call the same Amazon Polly API, manage access with AWS Identity and Access Management (IAM), use the output in your existing applications and contact-center flows, and monitor the workloads that synthesize speech for your customers. This launch is designed to elevate the speech output quality that you can hear with your existing Amazon Polly implementation. No changes to your code are required to get the upgraded voices. For applications that already use Amazon Polly Generative voices, the upgrade happens in place. For new applications, the same SynthesizeSpeech API gives you access to a larger set of generative voice catalogs. With the Bi-Di Streaming API StartSpeechSynthesisStream , you can stream text to Amazon Polly and receive synthesized audio back simultaneously. This is well-suited for real-time applications like voice agents and bespoke characters in games. Competitive price-performance The Amazon Polly Generative engine offers competitive pricing on a pay-as-you-go basis. With this model upgrade, you get improved voice quality and accuracy at the same price. There’s no cost increase, no new SKU, and no changes to your existing integration. You can cache and replay synthesized static prompts with no additional cost. Refer to Amazon Polly pricing for complete cost information. The Amazon Polly Generative engine now delivers more natural, expressive speech alongside low-latency streaming for real-time applications, Speech Synthesis Markup Language (SSML) support for fine-grained control, and the operational reliability of a fully managed AWS service. This combination of quality, performance, and cost efficiency supports a wide range of production use cases. What you can build Contact centers. Accuracy matters most in customer service conversations. When a caller checks an order, a voice that reads back the tracking number, address, and confirmation code correctly helps resolve the call in self-service rather than escalating to a human agent. Amazon Polly Generative voice’s accurate entity pronunciation and low-latency generation make automated interactive voice response (IVR) interactions sound more conversational. Cascaded voice agents. Using Amazon Polly in a voice agent framework, you can convert LLM response text into clear, natural-sounding speech and stream it back to users in near real time over WebSockets or telephony channels. This makes live conversational voice possible. The primary benefits are low-latency playback, high scalability, and cost efficiency. Media, localization, and dubbing. Convert scripts into finished audio for video, advertising, podcasts, product overviews, and audiobooks. You can use one voice or many, without a studio session. More natural intonation reduces post-production. The expanded locale coverage, including first-time generative Hindi and Finnish, means you can localize a single piece of content into more markets from the same pipeline. E-learning and training. Narrate courseware, onboarding, and knowledge-based articles at scale, and update them as soon as the source text changes. No re-recording is required. More expressive delivery supports engagement through long-form material. Accessibility. Provide a clear, natural spoken version of articles, documents, and applications so more people can consume your content by listening: read aloud, on the go, or with a screen reader. Interactive and transactional experiences. Voice-enable games, kiosks, IVR menus, and IoT devices with responses generated on demand. Use Amazon Polly when the voice needs to sound natural and read names, numbers, product details, addresses, and confirmations reliably. “At Sanoma Media Finland, we use Amazon Polly to turn our real-time news articles into audio, giving the listener the choice to listen whenever it suits them better. Today we publish over 10,000 articles per month using Polly’s Finnish neural voice. Based on our testing, Suvi’s new generative voice is a clear improvement over the previous neural voice – it sounds noticeably more natural and makes for a more enjoyable experience for the listener.” — Justus Reinikainen, Product Manager, Sanoma Media Finland Getting started You can access the new voices using the AWS Management Console , AWS Command Line Interface (AWS CLI) , or the AWS SDKs. Prerequisites: Active AWS account with Amazon Polly access. AWS Command Line Interface (AWS CLI) installed and configured. Python 3.10+. Boto3 installed: pip install boto3 . AWS Identity and Access Management (IAM) permissions: polly:SynthesizeSpeech . Here’s a quick example using the AWS SDK for Python (Boto3) with the SynthesizeSpeech API and a generative voice: import boto3 polly = boto3.client("polly", region_name="us-east-1") response = polly.synthesize_speech( Text="Hello! This is Amazon Polly's next-generation generative voice.", VoiceId="Kenneth", # any generative voice in the catalogue Engine="generative", OutputFormat="mp3", ) with open("speech.mp3", "wb") as f: f.write(response["AudioStream"].read()) You can also make the same request with the AWS CLI: aws polly synthesize-speech \ --region us-east-1 \ --voice-id Kenneth \ --engine generative \ --output-format mp3 \ --text "Hello! This is Amazon Polly's next-generation generative voice." \ speech.mp3 To retrieve the full, current voice list programmatically, call DescribeVoices . You can use this in deployment checks to confirm that a selected generative voice is available in the Region where your application runs. Availability The next-generation Generative engine and the new voices are now generally available. The 14 new Generative voices are available in the following AWS Regions: US East (N. Virginia), US West (Oregon), Europe (Frankfurt), Europe (Zurich), Europe (London), Asia Pacific (Sydney), Asia Pacific (Seoul), Asia Pacific (Singapore), Asia Pacific (Tokyo), Canada (Central), and AWS GovCloud (US-West). Getting started, by audience: Developers – Call synthesize_speech with Engine="generative" in your existing application. No other changes are required. Contact-center teams – The improvements reach Amazon Connect and Genesys flows automatically. Everyone – Access any voice in the Amazon Polly console . A few things to know: Standard and neural voices are unaffected. Speech marks are not available on generative voices (unchanged from today). If your application requires them, standard and neural voices continue to support them. Generative engine continues to support SSML tags. See supported SSML tags for more details. Use current voice availability at runtime. Call DescribeVoices or check the console when deploying across Regions. Conclusion The next-generation Generative engine of Amazon Polly is available today, delivering more natural and more accurate voices, adding 14 new voices, including generative voices generally available in Hindi and Finnish for the first time, and requires no change to your code. See Amazon Polly pricing page for pricing information. To get started, open the Amazon Polly console to try the voices, and see the Amazon Polly documentation to build with them. About the authors Suyin Wang Suyin is a Senior Generative AI Solutions Architect at AWS. She has years of experience building data-driven and machine learning applications that solve real-world problems. She enjoys helping customers identify the right problems to solve and build generative AI solutions that drive business value. In her spare time, she loves singing, or trying out new recipes in the kitchen. Gosia Brylowska Gosia is a Program Manager at Amazon IMAX Polly. With almost 20 years of experience in journalism, marketing and AV production & over 10 in program & product management (last 7 being in Amazon TTS), Gosia strives for excellency translating engineering goals into customer delights on a daily basis. Outside of work you can find her dog walking or watching Star Trek TNG for 100 th time… or catching waves &sightseeing with friends somewhere in Sri Lanka – you just never know with this one. Nikie Xiong Nikie is a Senior Technical Product Manager at Amazon Polly. She has 10+ years of experie
Generating multiple videos of a dynamic scene from a single reference video requires synthesizing both observed and unobserved content consistently across viewpoints. Existing camera-control methods for video diffusion typically rely on architectural mechanisms, including camera embeddings, cross-view attention, and geometric conditioning. Although effective for static scene content, these approaches often struggle with the fine-grained, dynamic geometry that must remain coherent across views and over time. We synchronize the diffusion process itself by sampling the Gaussian noise prior in a canonical 3D space derived from the source video and rasterizing it into target cameras, rather than sampling independently in each 2D image plane. Our method, SyncVid, lifts a monocular video into a canonical 4D representation, associates noise with scene geometry and unseen space, and rasterizes the resulting structured noise into arbitrary views. Because rendering can introduce local correlations, we further propose a cross-view-consistent whitening procedure that decorrelates neighboring pixels while preserving correspondence across views. The structured noise serves as the diffusion prior for a LoRA-adapted pretrained video diffusion model, requiring no architectural modifications. By enforcing spatiotemporal and cross-view coherence at the source of stochasticity, SyncVid generates synchronized videos and improves both perceptual quality and cross-view consistency in our experiments. Project page: https://qualcomm-ai-research.github.io/SyncVid-Webpage.
IskayPet’s office teams had a knowledge challenge. Answers to everyday operational questions, such as a return policy, a veterinary protocol, or an onboarding step, lived in hundreds of Microsoft SharePoint documents spread across four languages and three countries. Finding the right information in large document repositories took a long time. Every new store, country, or procedure made the problem bigger. Retrieval Augmented Generation (RAG) is the usual way to solve this. But running RAG in production normally means operating vector databases, embedding pipelines, and ingestion jobs yourself. In this post, we show how IskayPet, a leading pet care company in Iberia, built a corporate knowledge assistant on Amazon Bedrock Managed Knowledge Bases. The assistant answers employee questions across retail operations, veterinary care, people and onboarding, and procurement, in four languages and across three countries, with no RAG infrastructure to manage. The team built the whole solution, from the store-facing frontend to the AWS architecture and deployment, using Kiro with Claude Haiku 4.5. We cover the architecture, the SharePoint integration, the retrieval configuration decisions, and the results in production. About IskayPet IskayPet is a leading pet care company in Iberia. It operates more than 330 stores across Spain, Portugal, and France under the Kiwoko, Tiendanimal, Kivet, Clinicanimal and Animalis brands. With more than 4,500 employees and 7 million customers, IskayPet runs operations that span retail, veterinary clinics, animal care, procurement, and people and onboarding. Each area has its own policies and procedures, in Spanish, Portuguese, French, and English. The challenge Across every business area, IskayPet office staff searched hundreds of SharePoint documents by hand to answer operational questions about procedures, policies, contracts, and onboarding. With operations in three countries and four languages, this took real time, and the answer was often outdated or wrong. There was no reliable way to query corporate knowledge, and the problem grew with every country, area, or document added. IskayPet needed a solution that could do the following: Answer natural-language questions grounded in their own corporate documents. Work across four languages without maintaining separate systems. Route each question to the right body of knowledge (an area, a country) without cross-contamination. Scale to new areas and countries without re-architecting. Cite its sources so employees can trust and verify answers. Reuse their existing SharePoint repositories without a costly migration. Why Amazon Bedrock Managed Knowledge Bases IskayPet chose Amazon Bedrock Managed Knowledge Bases for its managed RAG infrastructure, which removed the need to run vector stores, embeddings, or ingestion pipelines. The main reasons were: Managed RAG: Amazon Bedrock handles data ingestion, multimodal parsing, vector storage, embedding generation, and retrieval. IskayPet pays only for what it stores and retrieves. Native SharePoint connector: The connector ingests documents directly from IskayPet’s existing SharePoint environment, which avoided a data migration. With advanced indexing enabled, it also processes content from images embedded in documents. Built-in hybrid search: Combining semantic and keyword (BM25) search at no extra cost recovers exact terms, such as product codes and references, that pure semantic search can miss. Managed reranking: Retrieved passages are reordered by relevance at no additional charge. Document-level access control: The knowledge base can enforce the source system’s own permissions at retrieval time, so a query returns only passages from documents the requesting user is already allowed to read. This lets IskayPet reuse SharePoint permissions instead of rebuilding an authorization model. Model flexibility: IskayPet can start with a cost-effective model and change it later without rewriting infrastructure. Native RAG evaluation: The team measured quality at each step with the built-in evaluation tools. Solution overview The following diagram shows the end-to-end flow, from the employee’s question through permission-aware retrieval to the grounded, cited answer. Figure 1: Solution architecture The solution is an assistant that employees reach through their existing channels. When an employee asks a question, the assistant retrieves the most relevant passages from its knowledge base, restricted to the documents the employee is allowed to read. It then generates a grounded, cited answer in the employee’s language. At a high level, the flow works as follows: An employee submits a natural-language question through the chat interface. The application calls the knowledge base for the requested assistant, passing the employee’s identity so that document-level permissions are applied to the retrieval. The knowledge base runs hybrid search (semantic and keyword) over the indexed content, reranks the results, and returns the most relevant passages with their source references. A foundation model (FM) generates a grounded answer from the retrieved passages. The prompt requires source citation and tells the model to say when the answer isn’t in the retrieved content. The employee receives the answer along with the source documents and can leave feedback (thumbs up or down) that feeds the evaluation set. Building the solution with Kiro and Claude Haiku 4.5 IskayPet built the whole solution, from the store-facing frontend to the AWS architecture and service deployment, using Kiro , the agentic development environment, running on Claude Haiku 4.5 on Amazon Bedrock. As new models become available on Amazon Bedrock, IskayPet plans to evaluate and adopt them for each use case. For model availability by AWS Region, see Supported models by AWS Region in Amazon Bedrock . Store-facing frontend – The team used Kiro to build the frontend that store employees use to interact with the assistant, generating and iterating on the UI code with Claude Haiku 4.5. Architecture and deployment – Kiro also helped design the AWS architecture and provision the services behind the solution: knowledge base configuration, data sources, AWS Identity and Access Management (IAM) roles, and the application layer. An agentic workflow with Kiro and a fast, cost-effective model like Claude Haiku 4.5 helped a small team move from concept to production quickly. It also kept the application code and the infrastructure consistent and maintainable. Ingestion: The SharePoint connector with advanced indexing IskayPet’s documents already lived in SharePoint, organized by area and country. Rather than migrate them, the team used the native SharePoint connector in Amazon Bedrock Managed Knowledge Bases. In the data source’s Advanced configurations, under Content indexing and Advanced indexing, IskayPet enabled Visual content in documents. This setting extracts content from images embedded in documents (PDF, DOCX, PPTX) and from standalone images. Figures and scanned content inside procedure documents became searchable without a separate preprocessing pipeline. The connector syncs incrementally, so new and updated documents are picked up on each sync. Multi-country, multi-area architecture A key design decision was how to structure knowledge for four languages, several business areas, and three countries. IskayPet weighed two approaches: A single shared knowledge base, with every document tagged by country and area and a retrieval filter applied per query. This maximizes reuse of shared content but couples all domains into one index. A dedicated knowledge base per assistant, each scoped to the relevant SharePoint sites or libraries. This keeps each body of knowledge isolated end to end and lets each assistant be tuned independently. IskayPet chose a knowledge base per assistant. Each assistant points at specific SharePoint sites and can be narrowed further to specific document libraries or folders through inclusion paths, so it indexes only the intended content and not the whole corporate site. Adding a new area or country means adding an assistant and pointing it at the right content, not re-architecting the system. Access control Access is governed by the users’ existing SharePoint permissions. The assistant doesn’t grant access to anything new: It only makes it more straightforward to query documents an employee could already open. This works on two levels. Which assistants a user sees – An employee is shown only the assistants whose underlying documentation they’re entitled to read, whether that entitlement comes from direct permission, group membership, or access to a specific folder. If access can’t be confirmed, the assistant is hidden rather than shown. Which documents inform an answer – Access is enforced again at retrieval time, per document: The employee’s identity is passed to the knowledge base, which returns passages only from documents that person is authorized to see. Amazon Bedrock Managed Knowledge Bases supports this document-level access control natively, so the same answer to the same question can legitimately differ between two employees with different permissions. Because permissions are read from SharePoint on every access check, revoking or granting access in SharePoint is reflected in the assistant without changes to the application. Retrieval and generation configuration The quality of a RAG system is decided mainly at ingestion, retrieval, and generation. Fine-tuning is a complementary technique. For this use case, IskayPet tuned the following settings: Chunking – Handled by the managed knowledge base, so the team didn’t build or tune a chunking pipeline. This removed a component that’s usually significant operational effort in a self-managed RAG. Hybrid search – The knowledge base combines semantic and keyword retrieval, which recovers exact terms such as product codes that pure semantic search can miss. Number of results – Tuned against an evaluation set. The assistant retrieves 25 passages and, after removing duplicates and fragments too short to add context, injects up to 15 into the prompt. This number is the main cost lever, because it multiplies input tokens on every question. Generation prompt – Enforces a corporate tone, requires source citation, and tells the model to say when the answer isn’t in the retrieved content, which reduces hallucination and adds traceability. As a production control, the team is adding Amazon Bedrock Guardrails to apply content filtering and responsible AI policies to the generated answers. Evaluation– Native Amazon Bedrock RAG evaluation, using the thumbs up and down feedback from real usage as an evaluation set to measure quality before and after each change. Generating the response For answer generation, IskayPet uses a cost-effective foundation model on Amazon Bedrock. Because the knowledge base decouples retrieval from generation, IskayPet can switch or upgrade the generation model without touching the ingestion or retrieval layers. This lets the team balance cost and answer quality as usage grows. Cost efficiency The knowledge base includes multimodal parsing, embedding generation, and reranking at no additional charge, so IskayPet’s ongoing cost comes mainly from index storage, retrieval API calls, and generation-model tokens. In production, input tokens dominate that cost, because of the passages injected into each prompt. The team is evaluating further optimizations as usage grows, including prompt caching for the stable part of the prompt and batch inference for non-interactive workloads. Results In roughly the first two months in production, IskayPet has seen: Faster answers – Time to find an operational answer dropped from 3–5 minutes to about 7 seconds. Higher productivity – Around 95 percent less time spent searching for procedures and policies. Scale – The solution is expected to handle roughly 100,000 queries per month across all business
What visual representation should a navigation policy act on? Navigation policies built on vision foundation models largely leave representation design to the encoders' pretraining objectives. We present RepresentNav, a navigation foundation model built on a self-constructed representation space: a visual representation constructed by the policy's own objectives from vision foundation models. RepresentNav builds this space with Representation Residuals, a compact three-layer transformer that attends over features spanning the full depth of multiple complementary vision foundation models, so that the policy learns to see what it needs for navigation. In the same space it dreams what it wants: before every action a latent chain-of-thought predicts the representation of the scene a moment ahead. The dream is trained end-to-end with the policy as a joint-embedding predictive architecture (JEPA), with a next-representation prediction loss and a regularizer on its targets. RepresentNav sets a new state of the art on VLNVerse, VLN-CE R2R, EVT-Bench, and two point-goal benchmarks, and its gains carry over to a physical robot amid clutter and moving people. On VLNVerse fine-grained, RepresentNav achieves 83.80% success, outperforming the previous state of the art by 20.05 percentage points, and on EVT-Bench distracted tracking it reaches 85.5% with a single camera (+10.1). With the same encoders, the self-constructed space clearly outperforms concatenating their outputs and learns to weight their layers unevenly, and a dream in it cuts collisions by a quarter. A single dreamed latent decodes into the depth, semantics, and 3D occupancy of the scene a moment ahead, more accurately than pretrained encoders applied to the real future frame, although occupancy is never supervised, and the rollout tracks the robot's own motion over the next second: the dream emerges as a compact latent world model.