Synthesizing believable 4D hand-object grasping sequences is important for character animation, extended reality, and embodied agents. Grasping requires a hand to approach, contact, and lift or manipulate an object coherently. Existing synthesis methods often model only the final static grasp or rely on additional conditions, such as language, target grasps, or object trajectories, to specify the interaction. We instead study the underconstrained problem of generating a complete grasping motion from a single initial hand-object state. Our framework decouples hand-motion generation from object-motion recovery. Given an object point cloud in the initial wrist frame and the initial parameterized hand configuration, conditional flow matching predicts a sequence-level contact map and a hand pose at each normalized sequence time. This time-indexed formulation avoids the error accumulation of autoregressive rollout while representing the complete motion. We then infer an attachment frame and recover rigid-object motion through training-free, contact-guided weighted registration. We evaluate on held-out DexYCB objects and unseen object geometries reconstructed from in-the-wild videos by Open-CHOIR. Compared with LatentHOI, our method produces smoother hand and object motion in every setting, matches or improves on its contact and penetration metrics in the wild under a shared initialization, and is about 30x faster per generated frame. In a 15-participant study, our motions were preferred in roughly two thirds of (forced-choice pairwise) comparisons. Precise contact closure on novel geometry, however, remains a key open challenge.
Text-to-motion generation typically relies on dataset-specific skeletal representations, with models trained separately for benchmarks with different joint layouts. We investigate whether image diffusion architectures can model 3D human motion through a common image-compatible representation that accommodates heterogeneous skeletal layouts while preserving motion-specific kinematic structure. To this end, we organize root-centered joint trajectories into topology-informed joint--time images and model global root dynamics through a complementary latent stream. A motion autoencoder conditions local-motion decoding on the root latent, while a text-conditioned Scalable Interpolant Transformer (SiT) jointly generates both latent streams, with supervision on decoded 3D joint positions further constraining generation in motion space. In addition to dataset-specific models, we train a single motion autoencoder on HumanML3D and KIT-ML, followed by a single text-conditioned SiT shared across both datasets, preserving their respective skeletal definitions without skeleton retargeting to a common joint layout. Experiments on HumanML3D and KIT-ML show that our dataset-specific models achieve state-of-the-art performance, while shared training yields further gains on both benchmarks, demonstrating the benefits of shared modeling across heterogeneous skeletal layouts.
Spatial intelligence, the capacity to represent and reason about physical space, underlies embodied perception and real-world visual understanding. Existing benchmarks probe it through spatial relations, perspective-taking, and mental modeling. What remains underexamined is whether a model can maintain a coherent scene representation across views, since pairwise comparison alone cannot reveal which view departs from the truth. We thus introduce Same Space or Not (SSoN), a benchmark of cross-view contradiction resolution comprising 1,000 questions from seven indoor scene sources and six categories of controlled 3D edits. Each question presents three views of an unchanged scene and one view of an edited variant. Models must identify the inconsistent view and explain the contradiction without a designated trusted reference or a marked target. Our pipeline iteratively translates human review into generation constraints. Automated checks and exhaustive human validation screen out both trivial visual shortcuts and questions lacking sufficient evidence for a unique answer. Humans achieve 83.3% accuracy, compared with 39.7% for the strongest vision-language models (VLMs), falling to 34.9% when a valid visual explanation is also required. We investigate prompting, camera information, and 3D reconstruction as interventions. On a 100-question intervention set with a 24% baseline, prompting and 3D reconstruction remain below 50% answer accuracy, except when ground-truth camera information is supplied, which reaches 61% answer accuracy but only 48% True solve accuracy. On a held-out 201-question test set, Qwen2.5-VL-7B achieves 21.89% accuracy, 33.33% after answer-only fine-tuning, and 41.79% after fine-tuning on both rationales and answers. These results identify cross-view contradiction resolution as a substantial weakness of current VLMs that can be partly addressed by targeted supervision.
中文 | English 新增功能 新增实验性思考过程机器翻译插件，可在插件页启用，选择 Bing、Google 或付费 DeepSeek Flash。 @tianyicui , @ZiyaZhang , @imccyu 新增实验性 Git Worktrees 插件，可在插件页启用，由 Agent 创建并进入独立的 Git checkout。 @tianyicui 新增实验性插件的 Session 状态记录接口，供插件写入和读取自己的会话记录。 @tianyicui 全局指令支持读取共享 agents 目录中的 AGENTS.md，可通过 DSH_AGENTS_HOME 指定目录。 @tianyicui 新增 working_directory 工具，并为 TypeScript/Python SDK 提供会话工作目录的读取和切换接口。 @tianyicui Official 插件列表新增 Claude Code 和 Codex 组合包的按需安装入口，发布版安装对应版本的包。 @tianyicui 新增工作步骤收起时机设置，可选择回答结束后收起，或在下一条普通消息到来时收起。 @imccyu pi-ai 路由支持按模型能力处理会话中的系统提示词更新，以及工具的动态新增和移除。 @Tuyixiang SSH 新增独立可执行的 helper 运行时，支持远端文件操作、进程、终端、沙箱和 Node PTC。 @LegGasai 问题修复 修复模型响应流停止输出、且底层传输不响应取消时，请求超过空闲超时仍一直挂起的问题。 @tianyicui 修复插件包元信息不可读，或辅助请求扩展准备失败时，整个 DeepSeek 请求被阻断的问题。 @turtle1999 , @turtle2099 修复凭据、配置和技能文件监听器关闭后的未捕获异常，以及 Agent 关闭时资源提前释放的问题。 @tianyicui , @kermanx 修复模型设置刷新期间仍可基于旧数据编辑或删除配置，导致刚保存的选择未出现在编辑器中的问题。 @tianyicui 修复部分 Windows 环境下，持久 PowerShell 在大量输出滚屏后迟迟无法结束当前调用的问题。 @Chinesezjc 修复 Windows 缺少 sleep 命令时，Claude Code Mods 的 Blast Radius 示例在取消后仍执行工具的问题。 @tianyicui 修复 Trajectory 将等待、流式输出或重试中的请求标为完成，以及压缩任务提前显示结果的问题。 @turtle2099 , @kermanx 修复后台工作流仍在运行，却因父级工具步骤或对话轮次正常结束而被标为已中断的问题。 @turtle2099 恢复可继续子代理输入栏的图片粘贴和拖放，支持运行中追加图片和完成后发送图片续发消息。 @ZiyaZhang 修复在 /goal、/plan 等高亮命令与参数交界处输入时，编辑器报错及草稿与屏幕文字不一致的问题。 @LegGasai 修复展开右侧栏后，会话标题栏中的后台任务列表被侧栏遮挡或裁切的问题。 @turtle2099 修复 Windows Desktop 图片预览遮罩及关闭按钮与标题栏重叠、侧栏按钮悬停圆角不一致的问题。 @turtle2099 修复开发者工具捕获网络响应时，调用者取消请求后，响应读取和连接仍未结束的问题。 @turtle2099 修复本地 SenseVoice 语音识别开启 VAD 后，录音开头被截断、首字或音节丢失的问题。 @yudshj 修复 Windows 安装器对磁盘根目录的校验，拒绝带重复末尾路径分隔符的根目录写法。 @tianyicui 修复 macOS 应用的小尺寸 Finder 图标，以及复制和替换应用提示中的图标损坏问题。 @07akioni 修复 Windows Desktop 终端退出后，PTY 后台工作进程未随之清理的问题。 @winewill 体验优化 通用设置新增字体选项，可分别调整界面正文、代码与工具输出、侧栏终端的字体和字号。 @turtle2099 语音输入新增麦克风选择，可在插件详情页选择输入设备，并查看所选设备的实时输入电平。 @yudshj Web 支持绑定指定的 IPv4/IPv6 地址，并通过 --tls-cert 和 --tls-key 配置原生 HTTPS 监听。 @oraluben Shell 和插件安装详情支持横向查看及复制完整命令；后台 Bash 调用可展开输入和启动回执。 @tianyicui 优化含超长源码行的文件预览性能，超长行使用无高亮文本展示，普通行继续进行语法高亮。 @turtle2099 优化自动会话标题生成，对同主题的追问、补充和继续请求保留已有合适标题。 @tianyicui , @ZiyaZhang 允许远程主机访问的 Web 配置使用页面内目录选择器，不再在服务器上弹出原生目录对话框。 @oraluben 侧栏 Fork 和子代理目录浏览新增迁移提示，遇到需要迁移的会话时引导用户先打开原会话。 @imccyu 其他变更 Python PTC 接入 Session 文件沙箱策略；自定义组合需提供 sandbox 和 sandboxPolicy 服务。 @tianyicui 移除工具展示的 both 混合模式，仅保留 native 和 ptc；使用旧模式的配置与 API 调用需要更新。 @tianyicui 移除 agent-instructions 插件逐行 dshHome 配置，指令目录改从进程的 DSH_HOME 或默认目录读取。 @tianyicui 统一子代理执行与完成通知，工具立即返回 child ID；自定义 SDK runtime 需支持 session/wait。 @Dudu-0223 Agent Team 消息改为直接投递目标 Inbox，移除独立 outbox、自动重试和重发去重；相关配置与结果字段需迁移。 @Dudu-0223 默认 SDK profile 从固定的编程 Agent 改为通用 AI Agent 身份，适用于一般任务。 @Dudu-0223 可选插件支持极简模式；Claude Code 和 Codex 组合包可用于 headless、SDK 和 ACP profile。 @turtle2099 更新运行时工具、MCP 协议、图片处理和客户端渲染依赖，纳入兼容版本的安全修复。 @tianyicui pi-ai 更新至 1.0.2，包含上游模型目录更新和适配器修复。 @tianyicui New Features Add an experimental reasoning translation plugin, enabled from the Plugins page with Bing, Google, or paid DeepSeek Flash as the provider. by @tianyicui , @ZiyaZhang , @imccyu Add an experimental Git Worktrees plugin that can be enabled on the Plugins page for Agents to create and enter separate Git checkouts. by @tianyicui Add Session state-record APIs for experimental plugins to write and read their own session records. by @tianyicui Load global instructions from AGENTS.md in the shared agents directory, with DSH_AGENTS_HOME available to select its location. by @tianyicui Add the working_directory tool and TypeScript/Python SDK APIs for reading and changing a session’s working directory. by @tianyicui Add on-demand installation of Claude Code and Codex bundles to the Official plugin catalog, using packages matching the running release. by @tianyicui Add a setting to collapse work details either when the response finishes or when the next regular message arrives. by @imccyu Support in-conversation system prompt updates and dynamic tool additions and removals on pi-ai routes according to model capabilities. by @Tuyixiang Add standalone executable SSH helper runtimes supporting remote file operations, processes, terminals, sandboxing, and Node PTC. by @LegGasai Bug Fixes Fix requests remaining stuck beyond the idle timeout when a model response stream stops producing output and its transport ignores cancellation. by @tianyicui Fix entire DeepSeek requests being blocked when plugin package metadata cannot be read or auxiliary request-extension preparation fails. by @turtle1999 , @turtle2099 Fix uncaught errors after credential, configuration, and skill watchers close, and premature resource release during Agent shutdown. by @tianyicui , @kermanx Fix editing or deleting model settings from stale data during refresh, including newly saved selections missing from the editor. by @tianyicui Fix persistent PowerShell calls failing to settle promptly after large amounts of output scroll in some Windows environments. by @Chinesezjc Fix the Claude Code Mods Blast Radius example executing tools after cancellation on Windows when the sleep command is unavailable. by @tianyicui Fix Trajectory marking waiting, streaming, or retrying requests as completed and showing results before compaction tasks finish. by @turtle2099 , @kermanx Fix background workflows being marked interrupted while still running after their parent tool step or conversation turn ends normally. by @turtle2099 Restore image paste and drop in continuable subagent composers, including images sent while running and follow-ups after completion. by @ZiyaZhang Fix editor errors and draft/display mismatches when typing between highlighted commands such as /goal or /plan and their arguments. by @LegGasai Fix the background job list in the session header being obscured or clipped when the right sidebar is expanded. by @turtle2099 Fix Windows Desktop image-preview masks and close buttons overlapping the caption, and inconsistent sidebar-button hover corners. by @turtle2099 Fix response reads and connections remaining open after the caller cancels a request while Developer Tools captures network responses. by @turtle2099 Fix clipped recording onsets and missing initial words or syllables in local SenseVoice transcription with VAD enabled. by @yudshj Fix drive-root validation in the Windows installer to reject root paths containing repeated trailing separators. by @tianyicui Fix corrupt small Finder icons for the macOS app and icons shown in application copy and replacement dialogs. by @07akioni Fix PTY worker processes remaining after a terminal exits in Windows Desktop. by @winewill Improvements Add General settings for independently choosing fonts and sizes for interface text, code and tool output, and sidebar terminals. by @turtle2099 Add microphone selection to voice input, with an input-device selector and live levels for the selected device in plugin details. by @yudshj Allow Web to bind to a specific IPv4/IPv6 address and configure native HTTPS listeners through --tls-cert and --tls-key. by @oraluben Support scrolling and copying full commands in Shell and plugin installation details, and expand background Bash inputs and launch receipts. by @tianyicui Improve previews of files containing overlong source lines, rendering those lines without syntax colors while regular lines retain highlighting. by @turtle2099 Improve automatic session titles by retaining adequate titles for same-topic follow-ups, additional details, and requests to continue. by @tianyicui , @ZiyaZhang Use the in-page directory picker in Web configurations that allow remote hosts, instead of opening a native directory dialog on the server. by @oraluben Add migration guidance to sidebar forks and subagent catalog browsing, directing users to open the original session when migration is required. by @imccyu Chores Apply Session filesystem sandbox policies to Python PTC; custom compositions require sandbox and sandboxPolicy services. by @tianyicui Remove the both mixed tool-presentation mode, leaving native and ptc. Update configurations and API calls that use the removed mode. by @tianyicui Remove the per-row dshHome setting from agent-instructions and resolve instruction directories from the process DSH_HOME or its default. by @tianyicui Unify subagent execution and completion notifications, with tools returning child IDs immediately. Custom SDK runtimes must support session/wait. by @Dudu-0223 Deliver Agent Team messages directly to the target Inbox, removing the separate outbox, automatic retries, and resend deduplication. Migrate affected configuration and result fields. by @Dudu-0223 Change the default SDK profile from a fixed coding-agent persona to a general AI Agent identity for general-purpose tasks. by @Dudu-0223 Support optional plugins in Minimal mode and allow Claude Code and Codex bundles in headless, SDK, and ACP profiles. by @turtle2099 Update runtime utilities, MCP protocol integrations, image processing, and client-rendering dependencies with compatible security fixes. by @tianyicui Update pi-ai to 1.0.2, including upstream model-catalog updates and adapter fixes. by @tianyicui Full Changelog: dsh-v0.2.1-alpha.1...dsh-v0.2.1-alpha.2 官方 Release：dsh-v0.2.1-alpha.2。请查看 Release notes、依赖和许可证变化。
Text-guided long-term human motion generation is essential for applications such as continuously operating digital avatars. Despite recent progress on short clips, existing models struggle with multi-segment instructions that involve anaphora, where constraints established in earlier segments must be preserved over time. In this paper, we introduce a novel task of long-term motion generation from such anaphoric compositions. To mitigate this, we present ELMA, an episodic long-term motion dataset with dense segment-level annotations that preserve cross-segment dependencies. Built via a fully automated collection and filtering pipeline, ELMA extracts high-quality, continuous 30-second clips from in-the-wild videos, capturing the episodic narratives necessary for long-range state modeling. Leveraging this dataset, we fine-tune state-of-the-art motion generation models to incorporate long-range context. Extensive experiments reveal fundamental shortcomings in current models when handling context-dependent instructions. We demonstrate that ELMA enables significant improvements in both global physical realism and semantic alignment, setting a robust new baseline for long-term motion synthesis.
Modeling human reactions to spatial audio requires understanding both what is heard and where it comes from. Prior work generates motion offline from complete acoustic and spatial input sequences. Humans, however, react to sounds without knowing what they will hear next. We present Sonic-AMG, to our knowledge the first framework for streaming human motion generation from spatial audio. The model encodes acoustic content and body-relative source geometry as separate tokens. Conditioned on these tokens, a causal transformer with a flow head generates continuous motion latents, from which a causal decoder reconstructs motion segments. Standard teacher-forced training derives motion histories and body-relative source locations from ground-truth motion, whereas streaming inference relies on generated motion for both, resulting in a training-inference mismatch. To reduce this history discrepancy, we introduce rehearsal training, which combines full-sequence teacher-forced supervision with supervised prediction along short autoregressive self-rollouts. To keep spatial conditioning consistent with generated motion, we propose spatial feedback, which recomputes body-relative source locations from generated root poses during both rehearsal training and inference. On SAM, Sonic-AMG achieves state-of-the-art FID and R-precision. It further supports real-time online generation of coherent reactions across sound events and source locations. Project page: https://jingchengni.com/sonic-amg/.
Installation You can install UniVRM using the UnityPackage or the UPM Package. The UniVRM supports Unity 2022.3 LTS or later. UnityPackage Download the unitypackage, and drag and drop it to import the package into your project. VRM 1.0: Download VRM 0.x: Download UPM Packages click to expand You can install these UPM packages via Package Manager in UnityEditor. Open the package manager window: Window -> Package Manager Click + at the top-right of the package manager window, then select Add package from git URL... Specify UPM packages using URLs below. [!Warning] Starting with version v0.131.0 , Assets/UniGLTF, VRM, and VRM10 have been moved to the Packages folder. Therefore, please specify path=/Packages/UniGLTF , path=/Packages/VRM , or path=/Packages/VRM10 . UPM package git URL com.vrmc.gltf https://github.com/vrm-c/UniVRM.git?path=/Packages/UniGLTF#v0.131.3 com.vrmc.univrm (VRM 0.x) https://github.com/vrm-c/UniVRM.git?path=/Packages/VRM#v0.131.3 com.vrmc.vrm (VRM 1.0) https://github.com/vrm-c/UniVRM.git?path=/Packages/VRM10#v0.131.3 You can also install via editing Packages/manifest.json directly. // Packages/manifest.json { "dependencies" : { // ... "com.vrmc.gltf" : "https://github.com/vrm-c/UniVRM.git?path=/Packages/UniGLTF#v0.131.3" , "com.vrmc.univrm" : "https://github.com/vrm-c/UniVRM.git?path=/Packages/VRM#v0.131.3" , "com.vrmc.vrm" : "https://github.com/vrm-c/UniVRM.git?path=/Packages/VRM10#v0.131.3" , // ... } } Development Samples click to expand via UnityPackage VRM 1.0 Development Samples: Download VRM 0.x Development Samples: Download via UPM Package You can find Samples in the Package Manager and then submit Import button. Window -> Package Manager -> Packages: In Project -> VRM-1.0 or VRM Release Notes 日本語 English Warning from v0.125.0 . com.vrmc.vrmshaders is removed. com.vrmc.vrmshaders is no longer required in manifest.json. What's Changed glTF/VRM 読み込み時のメモリフットプリント改善 by @hadashiA in #2824 [EditorExport][0x] VRM-0.X の export dialog から RuntimeTextureSerializer を選択できる by @ousttrue in #2820 NativeArray変換を ゼロコピー化して最適化 by @hadashiA in #2825 [VRMC_springBone_limit] fix clamp by @ousttrue in #2832 [VRMC_springBone_limit] fix editor ui by @ousttrue in #2834 [VRMC_springBone_limit] specVersion and validation by @ousttrue in #2833 UniVRM-0.131.3 by @ousttrue in #2835 [using漏れ] using UniGLTF.Extensions.VRMC_springBone_limit; を追加 by @ousttrue in #2838 Full Changelog : v0.131.2...v0.131.3 官方 Release：v0.131.3。请查看 Release notes、依赖和许可证变化。
Scenes Add living rooms, dining rooms, bedrooms, bathrooms and kitchens as top-level scene generators Add scene setups for beds, bathrooms, desks, dining tables, cocktail tables, sofas, wall storage and kitchens Refactor Infinigen 1 house layouts into furnished three-to-eight-room house scenes for Infinigen 2 Add indoor grid scenes for storage, desks, chairs, sofas, dining tables and cocktail tables Add standalone wall scenes for windows, painting grids, board shelves, storage shelves, cubbies, flush storage, doors and full-height windows Add collision-safe rotation variation to furniture arrangements Add collision-aware camera tours through at least three connected rooms in furnished houses Objects Add beds with varied frames, headboards, footboards, mattresses, storage bases, pillows and bedside furniture Refactor Infinigen 1 sinks, taps, bathtubs, toilets and wall-mounted bathroom hardware for Infinigen 2 Expand storage furniture with open, drawer, single-door, double-door and glazed-door fronts, including cabinets raised on legs Expand desks with plain, integrated-storage, top-storage and side-storage forms Expand dining and occasional furniture with circular tables, storage-bodied coffee and side tables, benches and new dining-chair variations Diversify sofas with real-scale proportions, sectional layouts, raised bases, varied arms, piped cushions and throw pillows Diversify floor, ceiling and table lamps with classic, vase-shaped, straight-leg, pedestal and square bases Refactor Infinigen 1 foliage into simple, broadleaf, ginkgo and maple leaf generators, phyllotactic plants and nursery-scale potted houseplants for Infinigen 2 Adapt vessel generators to create ribbed and twisted bowls and cups Add terrain generators for canyons, cliffs, mesas, mountains, rivers and volcanoes Add abstract, cauliflower, crystalline, flat-stone, ridged and natural-rock boulders Expand windows with rounded and rectangular frames, matching wall openings, curtains and shutters Expand doors with handles, glass panels and double-door variants Refactor Infinigen 1 television and monitor generators for Infinigen 2 Add rigged humans with procedurally fitted clothing and randomized body, hand, facial and eye-gaze poses Materials Refactor Infinigen 1 terrain and environment materials for sandstone, stone, cobblestone, cracked ground, soil, dirt, mud, sand, ice, lava and atmospheric haze for Infinigen 2 Add flaked-tile wall finishes with exposed underlayers Refine brick relief so it stays within the wall plane Expand wall-paint colors to darker, more saturated tones Refactor Infinigen 1 procedural graphic designs into tiled wall art for Infinigen 2 Add wrinkle effects for fabric, bedding, carpets and rugs Add wrinkle effects for paper Expand upholstery with unified cloth distributions, layered wear and wrinkles, and broader natural-dye leather colors Refine plastic surfaces with scale-aware relief and more visible grayscale grain Rendering and export Adopt true displacement as the default and include displacement assets in visual previews Standardize metric UV mapping for furniture and storage Standardize metric UV mapping for lighting Improve UV mapping for bowls and cups Improve UV mapping for handles Improve UV mapping for sinks, taps and bathroom hardware Preserve subdivision on instanced dining chairs Preserve subdivision on wall-mounted objects Preserve distinct object identities in repeated bathroom arrangements Preserve distinct object identities for wall bases Support explicit Cycles backend selection while automatic device selection is enabled Performance Reduce peak Cycles memory for a representative furnished room from 11.3 GB to 7.2 GB Eliminate multi-million-face door and handle assemblies while preserving their final shapes Reduce render geometry for furniture, curtains, windows, bowls, cups and flowers while preserving final shapes Limit render jobs to eight Blender CPU threads by default Report base and rendered triangle counts separately and identify the ten largest contributors when scenes exceed their triangle budget API and dependencies Require ProcFunc 0.37 and Shapely 2.x Expose common rendering modes and configuration types through the public exporter API Expose render-validation queries and a combined pre-render render_validity_check Split material shader-node validation into normal_input_used and unlinked_texture_vector Return total and per-object geometry estimates for multi-object inputs Standardize procedural mask presets on typed results Preserve complete generator results and object collections during traced scene composition Testing and CI Verify trace reproduction across object generators Extend visual regression coverage to procedural displacement Organize live and published visual comparisons by asset category 官方 Release：2.0.0a3。请查看 Release notes、依赖和许可证变化。
Motion retargeting is a critical preprocessing step for transferring human motion data to humanoid robots. While existing methods are effective for single-character motions, interactions involving multiple characters and objects introduce additional contacts and spatial relationships that must be preserved. Retargeting is commonly formulated as a weighted multi-objective optimization problem, where conflicting objectives can require careful tuning of relative weights and can still compromise important motion characteristics. We introduce PriMoRetarget, a prioritized motion-retargeting framework that replaces relative objective weighting with an explicit priority hierarchy. Lower-priority updates are constrained to preserve higher-priority objectives, allowing critical characteristics such as foot placement and interaction contacts to take precedence over lower-priority pose objectives. The method is evaluated on four datasets spanning single-character motion, character-object manipulation, and character-character interactions. The resulting motions better preserve task-relevant interactions and provide reference trajectories that can be tracked more reliably than those produced by widely used retargeting methods.
Recent motion generative models have demonstrated strong capabilities in synthesizing physically plausible character motion, but often overlook established animation principles used by professional animators to ground and design their animation work. Understanding and incorporating these principles into motion generative pipelines is essential for producing motions that serve not only physically grounded applications but also the needs of the character animation community. This enables the creation of characters that not only move in physically plausible ways but also feel alive, expressive, and engaging. To close this gap, we focus on the Exaggeration principle of animation and investigate how it can be incorporated into modern motion generative pipelines to produce more expressive character motions. To this end, we introduce a framework that operates at two stages of existing motion generative pipelines. The first stage introduces exaggeration during training, where we perform supervised fine-tuning of pre-trained text-to-motion models on our curated exaggeration dataset. The second stage operates at inference time, where we: (i) introduce a mathematical formulation of exaggeration based on dynamic movement primitives (DMPs); and (ii) leverage this formulation as an exaggeration guidance signal to guide existing diffusion and flow-matching text-to-motion generation models toward exaggerated motion without additional training. Through qualitative and quantitative evaluations against three strong motion generation models, we show that our methods generate more exaggerated and expressive motions while preserving neutral reference motion intent and physical plausibility.
Each GitHub release now publishes the matching fablecut package to npm automatically, with provenance, through npm trusted publishing. 官方 Release：v1.12.1。请查看 Release notes、依赖和许可证变化。
Added npm package. npx -y fablecut starts the editor and npx -y fablecut mcp runs the MCP server, so registering FableCut no longer needs a clone or a file path: claude mcp add -s user fablecut -- npx -y fablecut mcp . Installed from npm, projects, media and exports live in ~/FableCut ( FABLECUT_DATA_DIR still overrides), so updates never touch them. fablecut_patch_project takes an optional baseRevision : a patch planned against an older revision is refused with a conflict and the current revision instead of landing on top of a newer edit. 官方 Release：v1.12.0。请查看 Release notes、依赖和许可证变化。
Today, one in three pull requests on GitHub involves an AI agent . A year ago, that number was fewer than one in 10. If that pace holds, within the next two years, most of the code pushed to GitHub could be written by an agent. Much of it may never be fully read by a human. If developers and agents move faster, we have a responsibility to ensure protection keeps up with the accelerated rate of code creation. That means preventing more leaks before they happen and making the response to exposures that remain less dependent on manual human effort. This is a pivotal point for leaked secrets. Developers aren’t becoming more careless; they’re being outpaced. The tools that let developers create more software should also take on more of the work of protecting it. In this essay, I share the nine quarters of data behind that claim. I also introduce the fine-tuned classifier we built with Microsoft Applied Sciences to extend push protection to unstructured secrets. The model assesses a whole set of candidate secrets in less than two milliseconds and could more than double the number of secrets that we can prevent. Outpaced, not careless A new secret appears in publicly visible code about once every two seconds, doubling yearly for the past three years. Public discourse is quick to jump to the idea that AI made developers careless. Between Q2 2024 and Q2 2026, screened pushes grew 2.84 times while pushes carrying credentials grew 2.59 times . Across nine complete quarters of data, we found no statistically detectable trend regarding per-push prevalence. At the same time, we found data suggesting that, more than ever, developers understand the risk of accidental exposures and are less willing to accept that risk. Over the same period, the share of push-path blocks overridden by developers fell linearly from 6.63% to 3.93% . These figures challenge the common claim that agents are causing developers to become more careless. More pushes, no clear rise in push prevalence Public pushes Push prevalence 202M 574M · 2.8× 0 300M 600M 0% 0.5% 1.0% Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 2024 2025 2026 2026 Q2 · 574M pushes · 0.47% with secrets Public pushes, Q2 2024–Q2 2026. Push prevalence is the share with a detected secret. Covers supported provider patterns, including GitHub’s own tokens. At a fixed rate, doubling activity doubles expected exposures. If each exposure requires the same human response, the workload doubles too. The mean time to manually revoke a secret hovers around 40 days ; roughly one in five took more than 90 days . We’re accelerating the creation of software while exposed credentials can remain usable for weeks or months, because human remediation can’t scale at the same pace as development. Telling developers to be more careful cannot, on its own, solve that problem. As the amount of code grows, we must prevent more exposures and reduce human effort required by those that remain, if software development is to remain sustainable. Prevention scales with compute I’ve spent the past few years working on secret scanning at GitHub and the past year as product lead for the area. Our greatest impact has come from connecting the dots between detection and systems that can act. GitHub’s catalog covers more than 150 technical partners through our secret scanning partnership program. Through our partner program, we work with participating secret issuers to build out detectors and report public exposures so they can respond. In Q2 2026, public scanning successfully reported an average of 26 credential matches a second, including repeat observations. Once notified, a large number of these partners immediately revoke the token: OpenAI API keys, Google Cloud account credentials, Slack webhooks, Hugging Face user tokens, SendGrid keys, etc. The owner may still need to replace the token, but revocation can happen without waiting for a developer to find and process a GitHub alert. Push protection intervenes earlier. It stops recognizable credentials before they enter repository history, giving the developer or agent a chance to correct the change before there’s an exposure to investigate. We work with our technical partners to increase precision rates of their detectors as much as possible, until we’re confident enough to push-protect these secrets for the developer community by default. Thanks to the efforts of our partners, in the past month, a secret was blocked by push protection at least once every second. When it comes to issuer-bound credentials, GitHub blocks more secrets than those which slip. I’m proud of how ordinary we’ve made that feel for developers. Remediation scales with people When including additional secret types, push protection stops about 30% of newly detected secrets before they enter repository history. We find the remaining 70% after the credential is, unfortunately, already lost. And: Prevention scales with compute, but remediation still scales with people. Refusing a push costs compute; cleaning up a secret already lost to visible history costs a developer’s time and attention. As the amount of code grows, we must prevent more exposures and reduce the human effort required by those that remain, or else the volume of vulnerabilities introduced will become untenable. Telling developers to be more careful cannot solve this imbalance. Recognizing more of these secrets, earlier in development flows, is work the platform must take on. Solving the four-body problem Before a secret crosses the push boundary, the cost of stopping it is small, and the decision is binary: block or allow. After it crosses, the same string can authenticate to a real system, and the cost is unbounded. In many cases, our only detection clue may be the surrounding code and world context. A provider-issued token may have a recognizable prefix. An internal database password may be completely unstructured, with no identifying pattern at all. We were already using context to find these secrets post-push; the problem was balancing that context-aware judgement with other factors. We refer to this as the “four-body problem” for secret protection: precision, latency, throughput, and cost are coupled constraints. Prevention must be worth a developer’s time. A finding suitable for later review may not justify blocking a push. A false positive interrupts a developer and makes the next block harder to trust. A check that is too slow, expensive, or difficult to scale limits how often it can run. Protection at the push in under 2 ms GitHub’s AI-powered generic secret detection model uses surrounding code context to block password-like values in a database URL, Kubernetes Secret manifest, and Dockerfile, while allowing the placeholder changeme. Our new ModernBERT classifier assesses candidate secrets in context, without generating code or prose. It’s not only more precise than existing LLM-based pipelines, but it’s incredibly fast, evaluating candidate batches in under two milliseconds. It’s also extremely cost efficient, enough to run at scale in the critical path. The inclusion of our model in push protection makes it possible for us to more than double the number of secrets that we’re able to prevent. The feature is currently in private preview. Later this month, the feature will be available to organizations with GitHub Secret Protection across Enterprise Cloud and GitHub Teams. It will consume AI credits. We’re also bringing the model to developer surfaces beyond the push. Starting today, any organization with AI secret detection will be automatically updated to the new model. Alerts opened from these post-push scans remain included with an organization’s purchase of secret scanning at no additional cost. The model will also ship with GitHub Enterprise Server 3.23 in public preview , bringing AI-detected alerts to Secret Protection customers even in air-gapped environments. We’re adding the classifier to the /security-review command for the Copilot CLI and Copilot App , so Copilot users can address secrets before a push even without needing an organization’s GitHub Secret Protection plan. AI credit usage will be attributed to GitHub Secret Protection in your AI usage insights. Looking forward The future we want is one where developers can entrust more work to agents without supervising every request, and one where the number of people that an organization needs to keep its credentials safe no longer scales with the amount of code it writes. We owe the developer community the same progress in protecting software that we are delivering in producing it. We want people to build more software. Our capacity to protect it should grow with our capacity to create it. The post Secret protection must scale with software appeared first on The GitHub Blog .
At a glance Harnessed Agentic RL: Microsoft Research Asia introduces a training paradigm in which the same agent harness used in deployment participates directly in reinforcement learning, removing the need to reimplement the agent inside the training framework. Lightweight by design: Agent Lightning v1.0 delivers a complete agent RL control plane in roughly 3,500 lines of code. Native Kubernetes support: agents run as standard Kubernetes jobs on self-managed clusters, cloud Kubernetes, or local infrastructure, with no dependency on paid commercial sandbox services. Data-efficient training recipe: an end-to-end coding agent pipeline raised Qwen3.5-9B from 41.8% to 56.4% Pass@1 on SWE-bench Verified, a 14.6 percentage point gain, using only about 6,000 training samples based on open sourced dataset. AI agents have evolved from single models to complex full-stack systems built from models, tools, and execution environments. Their capabilities increasingly depend on the agent harness that coordinates them from outside the model. Reinforcement learning (RL) is an approach where AI systems learn through trial and error, guided by rewards and penalties for their actions. RL can make those agents better, but most agent RL systems require developers to reimplement the agent inside the training framework. That is costly, and it means the agent being trained is not quite the agent that gets deployed. To address this, researchers at Microsoft Research Asia have introduced the Harnessed Agentic RL training paradigm and open-sourced a fully rebuilt Agent Lightning v1.0 (opens in new tab) . Compared with the original, Agent Lightning, v1.0 puts more emphasis on staying lightweight, on integrating with real harnesses, and on a complete, reproducible agent RL training pipeline. Agent Lightning v1.0 was rebuilt around Harnessed Agentic RL, with key improvements: Lightweight : the entire framework is about 3,500 lines of code. Agent Lightning v1.0 implements a complete Harnessed Agentic RL system in a codebase that is small and clear enough to understand, modify, and extend. Training on a real agent harness : agents reach the model through the large language model (LLM) proxy in Agent Lightning v1.0, leaving existing harness code unchanged. Native Kubernetes support : agents run directly as Kubernetes jobs, without external commercial sandbox services. Self-managed clusters and local infrastructure alike can support rollouts at scale. A complete coding agent training example : an end-to-end pipeline built on Qwen3.5-9B raised Pass@1 on SWE-bench Verified from 41.8% to 56.4%, an absolute gain of 14.6 percentage points, using only about 6,000 training samples. The limits of traditional agentic RL Traditional agentic RL assumes the training framework owns the interaction loop with the environment. In a ReAct-style loop, the model generates an action, the environment returns an observation, the observation is appended to the context, and the model generates the next action, so the whole rollout maps onto one continuous token trajectory. Early RL systems such as verl, AReaL, and slime were built this way, which meant training an agent required rebuilding its loop inside the RL framework. Real harnesses have outgrown that assumption. Coding agents such as mini-SWE-agent, OpenHands, OpenCode, Claude Code, and Codex each bring their own context management, tool protocols, execution logic, and dependencies, as do general-purpose agent systems. Rebuilding one for training is expensive, and the rebuilt agent may no longer behave in the same way as the deployed agent. Agent Lightning takes a different route. It places an LLM proxy between the agent and the model. The agent continues to run as before: simply point the endpoint that previously called the model API at Agent Lightning, and the training framework can observe and record its model calls. In v1.0, the researchers go further and formally define this paradigm as Harnessed Agentic RL: whichever agent harness is used in deployment is the harness that takes part directly in reinforcement learning during training (Figure 1). Figure 1. Traditional agentic RL compared with Harnessed Agentic RL. In traditional agentic RL, the training framework manages the environment and the agent loop. In Harnessed Agentic RL, the harness manages both. Four challenges in training with real harnesses A core difference between Harnessed Agentic RL and traditional agentic RL is that the environment interaction loop is handled by the agent harness rather than the training framework. The training system can only observe a series of LLM request and response pairs, so a single rollout may be split into a variable number of training samples. This brings four key challenges: Retokenization and sample merging : Harnesses keep context as text, but RL training needs the token IDs sampled during the rollout. Passing text through the chat template and tokenizer again can shift token boundaries, so adjacent calls cannot always be merged into one sample. Advantage calculation : Retokenization, subagents and context summarization can split one rollout into several samples. Computing baselines and advantages directly at the sample level causes rollouts that produce more samples to be counted repeatedly, which alters the original statistical relationships at the rollout level. Loss normalization : Averaging loss by sample count gives more weight to rollouts that produce more samples. Since sample count is often just a product of harness behavior, loss normalization must also avoid being distorted by it. Training backend scheduling : Sample count and length are known only after the harness finishes, while GPU counts and data/tensor parallel configurations are usually fixed. The backend has to map a variable workload into fixed resources. Building a complete agent RL control plane in 3,500 lines of code In system design, Agent Lightning v1.0 treats simplicity as its first principle. The entire framework is about 3,500 lines of code, with three core components: the API Gateway, the Rollout Controller, and the Customized Trainer (Figure 2). The API gateway stores rollouts, models, and events, and serves as an OpenAI-compatible LLM proxy. It links every model call from the harness to its rollout and records the prompts, responses, and log probabilities that training needs. The rollout controller starts and manages agent execution, either as local processes or as standard Kubernetes jobs, keeping agent execution separate from the trainer. The customized trainer, built on verl, creates rollouts, waits for them to finish, collects samples, and assembles the final training samples through a sample adapter. As a result, for an existing agent harness, simply pointing the model endpoint at the Agent Lightning proxy is usually enough to connect quickly to RL training. Figure 2. The Agent Lightning v1.0 system architecture showing the API gateway, rollout controller, and customized trainer. Collocated async RL Rollout times vary widely across agents. Synchronous RL waits for the slowest agent in a batch and leaves GPUs idle, while fully asynchronous RL raises utilization but needs separate GPU pools for rollout and training. In response, Agent Lightning v1.0 introduces Collocated Async RL, which lets rollout and model updates share the same set of GPUs. Once the system has collected enough rollouts, the update begins: the API Gateway pauses accepting new requests and waits for requests already in progress to finish, and rollout resumes after the update completes. The entire state transition is transparent to the external agent harness. In experiments, this approach achieved about a 2x end-to-end speedup over synchronous RL while using fewer GPUs than conventional asynchronous RL (Figure 3). Figure 3. Synchronous RL, asynchronous RL, and Collocated Async RL compared. Collocated Async RL raises utilization while occupying fewer GPUs. Running agents on Kubernetes Collecting enough rollouts means running many agents at once, which consumes substantial CPU, memory, and compute resources. Other Harnessed Agentic RL frameworks often host those agents on commercial sandbox services such as Modal Sandbox or E2B, where cost climbs quickly with scale. Instead, Agent Lightning v1.0 runs them as standard Kubernetes jobs, reusing existing self-managed clusters, cloud Kubernetes, or local infrastructure (Figure 4). Existing compute resources are used more efficiently, large rollouts cost less, and the whole pipeline stays open source and reproducible. Figure 4. The Rollout Controller in Agent Lightning v1.0 provides native Kubernetes support, running agents directly as standard Kubernetes jobs. 6,000 training samples, a 14.6-point performance gain To test the approach, researchers built a full pipeline on SWE-smith, mini-SWE-agent, and Qwen3.5-9B, covering data cleaning, environment construction, reward-hacking safeguards, and RL training. The training set holds about 6,000 samples and needs no large-scale compute. RL training alone raised Qwen3.5-9B from 41.8% to 56.4% on SWE-bench Verified, a gain of 14.6 percentage points. The coding agent experiments further confirm the earlier analysis of two challenges: advantage calculation and loss normalization. Compared with sample-level handling, rollout-level advantage combined with rollout-level normalization achieves a higher validation reward and keeps policy entropy more stable during training (Figure 5). Figure 5. Pass rate and policy entropy for Qwen3.5-9B on the SWE-smith validation set. Technical report GitHub project Opens in a new tab The post Agent Lightning v1.0: A 3,500-Line Lightweight Agentic RL Framework for Training Agents with Real Harnesses appeared first on Microsoft Research .
Holistic co-speech animation is prone to averaging in both motion representation and speech conditioning. In coordinate-space diffusion, slow body posture, mid-frequency gesture strokes, and fast hand or facial details are entangled in one prediction target, often producing low-variance, over-smoothed motion. Meanwhile, dense rhythmic and acoustic cues can dominate sparse content-specific information under fixed multimodal fusion. We present DynaConTalk, a wavelet-constrained diffusion framework for long-form and controllable holistic co-speech motion generation. Diffusion operates in stationary wavelet transform (SWT) coefficient space, whose temporally aligned bands separate coarse posture evolution, gesture strokes, and fine expressive details. Our dynamic gating network preserves HuBERT and speaker identity as a base and selectively adds rhythm, mel, and transcript features through motion-state- and noise-aware residual gates. Attention pooling and learned depth routing deliver complementary conditions to each denoising stage, while a frame-resolution rhythm path preserves precise timing. A signed proposal-consensus update then reconciles these conditions with the evolving motion state. Matched-noise constraint injection uses the same sampling interface for history continuation and localized keypose repair, and extends to reference-guided control. Separate body-hand and facial denoisers, followed by inverse SWT and a pose-driven root regressor, produce holistic motion. Experiments evaluate generation quality, facial accuracy, temporal continuity, and controllable editing. Code, models, and the interactive editing interface are available at https://github.com/zhuyifeiabcd1/DynaConTalk.
Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision-language model scores a few rendered snapshots, or a language model reads the source. On worlds written by five frontier models we find that the two views disagree on 32% of required items, mostly code that no frame shows, and that fixed views miss small close-up contents. We present WorldBench, a benchmark and judge for open-ended, LLM-generated Three.js worlds. From one prompt describing a floating voxel island with ten biomes, physics, and day/night and seasonal cycles, the judge explores the running world, controlling its clock, orbiting it, and sending a navigator agent to frame each biome, and reads the code for what it sees. Neither channel is trusted on its own: a code quote counts only if it is text the source contains, and visual claims are checked against measured pixels where the property is measurable. A mutation test, in which we remove features by construction, shows that code-only judging gives full credit to four of five removed features, because their code remains in the file. Our judge cuts the points kept on removed features by a third (5.44 to 3.55 of 7.11), and what it still credits is mostly code that exists but never runs. We evaluate five frontier models: Claude Fable 5.1, GPT-6 Astra, Kimi K3, Grok 4.7 and Gemini 3.1 Pro. Code, prompt, tests and judge configuration are available at https://github.com/KrishBakshi/worldbench
## What's Changed - Choose installed fonts for text and captions, adjust letter and line spacing, style text backdrops, and preview against a solid black canvas (#198). - Publish completed video exports through Upload-Post after reviewing and confirming the account, platforms, title and exported file (#188). - Configure Cheaper Inference as a named LLM provider (#194). - Keep long Agent runs going beyond 100 tool calls, including parallel calls and continued text output, without the previous retention-limit failures (#186). - Keep MCP connections alive for same-project recovery, and reuse unchanged active edit drafts without losing staged operations (#195, #196). - Accept streamed tool calls whose type field arrives empty. - Preserve transcript-driven audio during rate stretching and use the explicitly selected source for slip edits (#189, #190). - Preserve media that may still be referenced by unreadable project snapshots (#191). - Restore the Export action when reopening a completed export dialog (#192). ## Upgrade Notes SQLite is now the only project-store backend: startup imports existing JSON project data, chats, history and generation jobs automatically, and leaves the original files untouched as backups. Project data stays in its current directory; moving it through Settings is unavailable in this version, and the JSON backups contain only pre-upgrade data, without subsequent edits. ## 中文更新 - 文字和字幕支持选择已安装字体、调整字间距与行间距，以及设置文字底板样式，预览画布默认改为纯黑色（#198）。 - 支持通过 Upload-Post 发布已导出的视频，发布前需核对并确认账号、平台、标题和导出文件（#188）。 - 新增 Cheaper Inference 模型供应商（#194）。 - Agent 长任务可继续执行超过 100 次工具调用，并行调用和后续文字输出不再受原有保留上限影响（#186）。 - 编辑器绑定失效后保留 MCP 连接，支持重新连接同一工程，并复用工程未变化的编辑草稿及其中的操作（#195、#196）。 - 修复流式工具调用的类型字段为空时导致任务失败的问题。 - 文字稿驱动的音频不再允许拖拽变速，滑移编辑会使用明确指定的源素材（#189、#190）。 - 历史快照无法读取时，素材清理会保留可能仍被这些快照引用的文件（#191）。 - 重新打开已完成的导出对话框后，可再次点击导出（#192）。 ## 升级提示 工程库统一使用 SQLite，启动时自动导入已有 JSON 工程、聊天、历史版本和生成任务，原文件保持不变，仅作备份。 工程数据保留在原目录，本版本暂不支持在设置中迁移工程目录；JSON 备份只包含升级前的数据，不会同步升级后的编辑。 ## Contributors Thanks to @rsmith4321, @aiapienthusiast, @jnvhr and @mutonby for their contributions to this release. ## New Contributors * @aiapienthusiast made their first contribution in https://github.com/0xsline/OpenChatCut/pull/194 * @jnvhr made their first contribution in https://github.com/0xsline/OpenChatCut/pull/196 * @mutonby made their first contribution in https://github.com/0xsline/OpenChatCut/pull/188 **Full Changelog**: https://github.com/0xsline/OpenChatCut/compare/v0.2.15...v0.2.16
People communicate how things should move by combining words with demonstrations: "open it like this." We present an interaction technique that brings this expressive resource to conversational 3D authoring. Building on "Put-That-There," our system combines speech, pointing, and spatiotemporal demonstrations to specify editable behavior. A hand movement supplies evidence for a mechanism's axis, pivot, range, and pace; an animated preview makes the interpretation inspectable. Users refine behavior through further words or demonstrations, and the system can request a demonstration to clarify intent. In a twelve-participant study comparing three input configurations, our system achieved 89% participant-declared completion versus 47% with speech alone, had the highest observed match rates on six categorical accuracy measures and tied on two, and was preferred by nine participants. This work makes demonstration part of an ongoing authoring conversation: behavior can be shown, inspected, and revised.
Language-guided panoramic video generation benefits various downstream applications, such as interactive 3D scene exploration, virtual reality experiences, and embodied agent training. Existing panoramic generators follow predefined trajectories, and interactive world models act through low-level actions in perspective views. We propose SPW-Nav, a streaming panoramic world model that understands movement instructions and streams one minute of 2K 360-degree video in real time from a single panorama. SPW-Nav interprets each instruction in the previously generated panorama as camera motion. Spherical rotation decoupling applies rotation exactly on the sphere, pose-aligned conditioning keeps translation inputs bounded over long streams, and a multi-term memory with a few-step generator continues the scene as instructions change. We also build SPW-NavSet, panoramic videos with camera trajectories and verified instructions. Driven by language, SPW-Nav outperforms prior panoramic generators in camera-following accuracy and video quality, and supports on-the-fly instruction switching.
Modern video models can generate realistic videos from real appearance references and proxy renders that specify scene structure, viewpoint changes, and motion. Evaluating this render-to-real capability requires a real target video depicting the same scene evolution, paired with an editable, geometrically registered 3D replica. Such data has traditionally required substantial manual modeling, calibration, and animation effort. We introduce RenderBench, a benchmark of 12 reconstructed real-world scenes spanning large-scale indoor environments and egocentric viewpoints, with both static and dynamic settings. Our construction pipeline combines visual geometry, neural reconstruction, and assisted 3D authoring. Each scene is decomposed into static objects and dynamic actors, registered to the capture cameras, and accepted only after multi-view geometric and temporal validation. Each evaluation unit contains appearance reference images, a held-out real target video, an editable digital twin, a matched proxy render, and renderer-native scene annotations. We evaluate transfer models against paired real target videos, retain PAI-Bench-C-compatible structural projections, and use scene annotations to localize failures by object, visibility, articulation, and motion. The first release retains 12 of 14 registered samples (85.7%), comprising 1,496 paired real-proxy frames. All released scenes pass file-integrity and environment-edit audits, while proxy diagnostics yield a depth si-RMSE of 0.2170 and instance mIoU of 0.3673. RenderBench provides paired real observations and editable scene state for assessing both appearance fidelity and preservation of geometry and dynamics.
Jennifer Neville is a partner research manager at Microsoft who’s built a career around understanding and advancing AI for real-world use, and much like the human-AI interactions she’s been studying, her early-career path was multiturn: math, then physics; cognitive science, then work; and finally computer science—despite her best efforts to avoid the field. In this conversation with Principal Applied Scientist Chad Atalla , she explores the role evaluation plays in pushing the performance boundaries of today’s AI systems to meet user needs and the “surprising failures” that emerge when models are tested beyond traditional benchmarks. Neville also shares practical guidance for working with current AI systems and discusses why looking closely at data matters when results defy expectations, and what decades of AI progress have taught her about predicting what comes next. From an unexpected career trajectory to the frontier of AI interaction and learning, this episode asks a larger question: what can we learn when the path—whether human or artificial—doesn’t unfold the way we expect? Subscribe to the Microsoft Research Podcast : Apple Podcasts Email Android Spotify RSS Feed Transcript [MUSIC] JENNIFER NEVILLE: There were times that I felt like I was spinning my wheels. Things weren’t working. I would get, kind of, dejected. And then we’d get to a point where we did learn something, and it was just the, the emotional thrill of that was, that was really what hooked me. The being able to, kind of, understand something that no one else understood yet … CHAD ATALLA: Sure … NEVILLE: … because it’s just at the frontier of what we know about things. STANDARD INTRODUCTION: This is the Microsoft Research Podcast , where Microsoft researchers—driving advancement through fundamental science and technology research—explore the who, how, and what’s next in computing and AI. [MUSIC ENDS] CHAD ATALLA: Hello, and welcome. I’m Chad Atalla, an applied scientist here at Microsoft Research. Today, I’m joined by Jennifer Neville, a partner research manager at Microsoft Research and the Samuel D. Conte chair professor of computer science and statistics at Purdue. Her research examines machine learning and AI for interactive domains and structured data, looking at how the data points that AI systems are trained on affect their behaviors and how that aligns with what users actually want. Across that research, she has published more than 130 papers with over 10,000 citations and received honors such as a National Science Foundation CAREER Award, a spot on IEEE’s “10 to Watch” in AI list (opens in new tab) , and best paper awards from the International Conference on Data Mining (opens in new tab) and the International Conference on Learning Representations . What amazes me most about Jen’s work is her foresight and ability to have a through line that stays consistent as the field of AI develops, and I’m excited to hear more about how she got to where she is today. Jen, thank you for joining me. JENNIFER NEVILLE: Thanks for having me. ATALLA: Awesome. Well, I would love to learn about how you got here today. And let’s rewind way back. When I was a kid, I wanted to be an astrophysicist, but of course, here I am with a computer science background. When did you know that you wanted to go into computer science? NEVILLE: That’s a good question. I … when I was growing up, I wanted to do anything but computer science because that’s what my dad was in, and I wanted to do anything but what my dad did. So when I went to college, first I majored in math and then in physics, but I really just couldn’t vibe with those majors. So I dropped out of college for a while, and when I went back to college again, I majored in cognitive science instead, which also was too squishy for me. Didn’t have enough math in it. At that point, if somebody had told me, “AI is the thing you should be doing because it combines the cognitive science with the math and computational thinking,” I think I would have saved myself a lot of time, [LAUGHTER] but that didn’t happen. And so I worked for a while. And then when I went back to school, I decided to major in computer science because what I wanted to do was think about working with data and dealing with data. And that’s when I found AI. Just by happenstance. I didn’t even really realize that it was part of computer science. ATALLA: Awesome. Well, it’s one thing to go into computer science and want to work on data or AI and another thing to want to be involved in research on that front. So what sorts of questions or big ideas sparked your research drive? NEVILLE: Yeah, that’s actually an interesting story, as well. I only got into research because I was in the honors program in my computer science degree, and as part of the honors program, you have to do a research project [LAUGHTER]. ATALLA: It’s mandatory, yeah. NEVILLE: It’s mandatory. And so when I talked to professors about what the research project should be, they actually said, “Well, you have to decide what topic you want to work on.” And at that point, I was interested in data and I was interested in AI, and I thought a lot. I did a lot of reading. And what I decided that I wanted to investigate was how to do data mining on web data that was interconnected, and so that … when I decided that was the question I wanted to work on, I got pointed to a particular faculty member … ATALLA: Nice. NEVILLE: … who had just started working in this nascent field at the time that was called statistical relational learning. And we did my project in that, I published a paper at a workshop, and I was, kind of, hooked. ATALLA: OK, yeah. NEVILLE: So I hadn’t planned to go on to grad school, but that experience … ATALLA: Yeah. NEVILLE: … made me want to go on to grad school and continue. ATALLA: Nice. What part of it do you think hooked you? Was it, like, the thrill of doing the research? Was it the environment of the conference and what academic publishing looks like? NEVILLE: It was really the thrill and the process of doing research. So it was a long project. There were times that I felt like I was spinning my wheels. Things weren’t working. I would get, kind of, dejected. But my adviser would be, kind of, like, you know, supportive and positive, saying, “No, keep going. We’re learning something.” And then, and then we’d get to a point where we did learn something, and it was just the emotional thrill of that was, that was really what hooked me. The being able to, kind of, understand something that no one else understood yet … ATALLA: Sure. NEVILLE: … because it’s just at the frontier of what we know about things was, uh, it’s just, it’s like a drug almost, right? [LAUGHTER] So it’s kept me in research for this long, that same, that same … chasing after that same feeling. ATALLA: Gotcha. Well, love it. Yeah, you joined Microsoft in 2021 from academia. And in fact, you’re still a professor, and you’ve been teaching and advising for 20 years. What motivated the shift to part of your professional life being research in industry , and how would you characterize the difference between industry research and academia research? NEVILLE: My research, kind of, spans the spectrum from theory to application. When I did my first sabbatical after I got tenure, a lot of colleagues that I had at the … that were at the same point in their career went off into industry labs for their sabbatical and never came back to Purdue. I thought about which direction I’d want to go, either more theory or more applied, and I thought at that point in my career, it’d be better to explore the theory side because then I might actually go back to Purdue. And so I went to the Simons Institute in Berkeley for my sabbatical, and that was very theoretical. It was a great experience, but it was kind of seamless to transition back to academia. On my second sabbatical, I went the other way, which was to come to MSR [Microsoft Research] and do a sabbatical here. And of course, being able to see how algorithms in theory, kind of, hit the—where the rubber hits the road with respect to how they behave in practice and real systems and with real users and real data, that’s hard to turn back from. And so that’s now why I’m still here. ATALLA: Gotcha. NEVILLE: So I think the difference between research in academia and industry depends on … really is affected by the target of where, where you’re aiming the research. And so I think fundamentally, it feels very similar, the questions you would ask in academia and industry, but in industry, your ability to apply it at scale in real systems on real data is really very different from academia, and in academia, I think you end up asking questions that are more abstractions that cover applications across a lot of different domains at once. And that’s how you get funding from places like DARPA [Defense Advanced Research Projects Agency] and NSF [National Science Foundation]. But in industry, it’s a little easier to just, you know, actually get your hands dirty and, and do it in the real systems. And then your, kind of, target is the products and the company’s interests. And so as … that sort of changes, maybe the types of questions that you would ask or investigate. So I think it’s actually great to be … ATALLA: Yeah. NEVILLE: … in both places, right? Like, there’s lots of synergies across the two, and I think there’s lots of opportunities to go to industry and come back to academia or to go start in industry and go to academia. But right now, if you’re doing work on AI systems, I think that industry is really the place to be. ATALLA: Gotcha. Yeah. Perhaps useful advice to folks who are deciding which way to take that decision in their career right now. I’ve been here at Microsoft for a little over six years and have been working on AI evaluation for much of that time, and there are, of course, a number of daunting fundamental questions and challenges in the space. And I initially came across your work and became aware of your work because part of your work proposes some solutions to some of these problems and helps to point out these problems. And my immediate reaction was relief that we have someone wonderful like you working on these things, and you’re going to take care of it for us. And I really appreciate how you balance critiquing AI evaluation and also providing solutions along the way. But I understand that that’s just a small sliver of your broader research agenda. And so how would you describe the current research charter for the team that you’re leading here? NEVILLE: Our team is called the AI Interaction and Learning team . What we really focus on is trying to push the frontier of behavior of these AI systems in realistic work environments. And so what that entails is studying where’s the boundary of the performance of the current systems, how do users experience that in practice with real workflows, and then how do we improve that and push the performance of the models for these kind of tasks—complex tasks—that users are working on. The reason we focus on evaluation is that we find that the standard way that ML and AI people evaluate are through benchmarks that are fairly simple with respect to how people would actually use them in practice. And so we find that the first thing that you really need to do is ask, what do we want out of these systems? And design practical evaluations to put the systems in those kind of environments. So that means things like multiturn behavior, collaborative environments, long-horizon tasks that users are doing. And then once we can see where the performance gaps or problems that happen in those evaluations [are], then that also gives us the knowledge of where, sort of, theoretically or algorithmically do we need to improve these systems. So we start with evaluation, but ultimately, what we’re trying to do is develop better algorithms, models, estimation methods … ATALLA: Sure. NEVILLE: … to push the performan
Agentic code review is becoming an essential piece of how development happens. It helps you inspect pull requests, catch issues, and decide what deserves attention before code ships. But the quality of existing AI reviewers can be hard to measure, and you need to know the strengths of a reviewer before you know if it will help you. Some reviewers surface more issues, some produce less noise, and some are stronger at catching critical problems while others surface smaller improvements, too. You may need code review to do different things within your workflow. That makes it important to understand how reviewers actually compare: what different systems catch, what they miss, and the tradeoffs they make. A good code review benchmark should reflect the diversity of real pull requests, capture a broad set of review findings, and support meaningful breakdowns by severity, category, and precision-recall preferences. For teams building code review agents, the benchmark should also provide an offline signal that reliably tracks whether changes are likely to improve the experience in production. Existing benchmarks often make tradeoffs between label quality, coverage, and how well they represent real-world code review, leaving a gap for a rigorous and reproducible evaluation methodology that brings these pieces together. We built ReviewBench , a new code review offline benchmark , to address that gap, and it is available for you to use today. It follows the language, repo size, and size distribution of pull requests, modeled after over 100 million real pull requests on GitHub. It uses a multi-source golden set and a consistent evaluation rubric and has been independently validated by senior engineers. Just as important, with the help of ReviewBench, our offline evaluation of Copilot code review (CCR) has become more effective at anticipating the direction of production experiments, giving us greater confidence that measured improvements reflect meaningful gains for users. In this post, we’ll walk through how ReviewBench is constructed, how it establishes reliable ground truth and scoring, and how to onboard your own code review system and submit results. Definitions of terms used in this blog post Benchmark : A standardized evaluation that tests code reviewers on a common set of pull requests using the same scoring methodology. Finding : A specific issue surfaced during code review. Golden set : A validated collection of known findings for each pull request, used as a reference for evaluating what a reviewer catches or misses. Precision : Of the issues a reviewer surfaces, the proportion that are valid. Higher precision generally means less noise. Recall : Of the known valid issues, the proportion the reviewer finds. Higher recall means broader coverage. F1 score : A single score that balances precision and recall equally. Fβ score : A variation of F1 that lets you put more weight on either precision or recall, depending on your review preference. ReviewBench at a glance 1 What we built A realistic, comprehensive benchmark for AI code review agents 103.9M GitHub pull requests Analyze distributions by language, repository size, and change shape. Representative benchmark corpus 219 public pull requests across 19 languages, aligned to GitHub-wide distributions while preserving substantive review cases. Multi-source golden set Human reviewers Frontier LLMs Static analysis Structured findings Every finding is labeled for severity and category, enabling user-tailored slices. Severity Critical Medium Low Category Correctness Security Reliability Maintainability Testing ...... Evaluation metrics Four metrics measure both known and newly discovered issues. Grounded precision Grounded recall Augmented precision Augmented recall Objective evaluation Measure improvement and compare across agents objectively. Help users choose the reviewer that fits their needs the best. 2 How we keep it trustworthy An auditable chain from rubric to expert validation and production checks Published rubric One explicit standard for all findings. Human-labeled dev set Senior engineers establish ground truth. Calibrated grader Aligned with human judgment. Uniform labeling Same standard across all sources. Published agreement Expert audit of benchmark quality. Auditable end to end 96.6% agreement Senior engineers independently labeled golden true-positives before release. Offline signals that anticipate production Benchmark movement is checked against online experiments. Improvements tend to show up online Regressions tend to show up online too How ReviewBench works Our benchmark is built around five principles: 1. Representative pull requests, not a demo set We analyzed 103.9 million GitHub pull requests to characterize the real-world distribution of code review workloads. ReviewBench contains 219 pull requests from 187 public open source licensed repositories spanning 19 languages, with its language and repository-size distributions closely matching GitHub overall. The complete benchmark dataset is publicly available. We make one deliberate adjustment to this distribution: while language and repository size mirror GitHub directly, pull request size is weighted toward the reviewable middle and tail. This reduces the overrepresentation of tiny, single-file changes while preserving more substantive, multi-file pull requests where review quality matters most. Quick corpus snapshot: 2. Broad ground truth discovery, independently judged No single reviewer, whether human or model, can identify everything worth finding in a pull request. To build a broader and more reliable golden set for ground truth findings, we follow a three-stage process: Gather candidate findings from diverse sources. We collect findings from real human reviewers, issues inferred from author follow-up commits, deterministic analysis tools, and multiple frontier LLMs across model families. Semantically deduplicate overlapping findings. We merge findings that identify the same underlying issue, broadening coverage without allowing agreement across producers to artificially inflate the golden set or making it dependent on any one source’s blind spots. Validate findings under a shared rubric. The source of a finding does not determine whether it is correct: a finding counts as a true positive only if it is true, relevant, and non-trivial. We use Claude Sonnet 5 as the LLM grader, applying a consistent evaluation rubric across all submissions. For transparency and reproducibility, we publish both the evaluation rubric and the judge used to apply it. 3. Metrics that measure both known and newly discovered issues Most benchmarks report precision and recall against a fixed golden set. ReviewBench reports six metrics in two families: Grounded precision, recall, and F1 score use only the existing gold-set labels. They provide the strict, apples-to-apples comparison: of the issues we already know about, how many did the agent find, and what share of its findings matched a known issue? Augmented precision, recall and F1 score also evaluate findings that do not match anything in the golden set. The judge independently determines whether those unmatched findings are true or false positives, allowing a reviewer to receive credit for valid issues that no producer in the golden set surfaced That distinction becomes more important as review agents become more capable. A fixed golden set inevitably becomes incomplete as systems discover issues its creators did not anticipate. Augmented metrics let ReviewBench recognize that behavior rather than automatically penalizing it. Because augmented recall expands the denominator based on what each agent discovers, we use grounded recall as the headline cross-system comparison and augmented metrics as an additional per-system diagnostic. 4. Configurable evaluation for different review preferences There is no single universally optimal review experience. Some developers may want to focus only on critical issues, while others also value lower-severity, non-breaking findings. Some prefer broader coverage, while others prioritize precision and minimal noise. Others may have specialized needs, such as security- or privacy-focused review. ReviewBench lets results be sliced by severity and category, while precision and recall capture different operating preferences. Users can also adjust β in the Fβ score to place more weight on recall for broader coverage or precision for lower noise. As these preferences change, the leaderboard is re-ranked accordingly, helping users identify the systems that best match their review priorities. 5. Internally audited and reproducibly evaluated Before release, we asked senior engineers who had not participated in building the benchmark dataset to independently re-label every ground-truth finding from scratch. Their true/false-positive judgments agreed with ReviewBench 96.6% of the time. We version the benchmark dataset, judge, and matcher used in every evaluation, so results can be compared under the same benchmark configuration and revalidated when the benchmark changes. We also publish the validation methodology, agreement measurements, and known threats to validity, so readers can see how benchmark quality is assessed and where uncertainty remains. Explore ReviewBench ReviewBench’s research preview version is now available through the ReviewBench website , where you can explore the full benchmark, compare code review agents, and bring your own agent to evaluate and iterate. With ReviewBench, you can: Explore the full benchmark dataset. The complete ReviewBench dataset is publicly available, including the pull requests, findings, labels, severity and category annotations. This allows you to inspect exactly what systems are evaluated on and reproduce benchmark results. Compare systems on the leaderboard. Results from evaluated code review agents using the full benchmark data are published on a common leaderboard, with views across overall performance, severity, category, and different precision–recall preferences. Bring your own agent and hill-climb. The full benchmark dataset, evaluation methodology, LLM judge prompt, judge model configuration, and self-serve runner are publicly available, so you can evaluate your own code review agent, inspect its strengths and gaps, and iterate against the same benchmark configuration. How we’ve used ReviewBench We have used ReviewBench to evaluate Copilot code review (CCR) across successive iterations, giving us a consistent way to measure progress, catch regressions, and prioritize promising changes. Over time, this has helped us improve the product. One of the most valuable benefits of ReviewBench is that it provides an early offline signal of how a change to the product is likely to perform in production. Across experiments evaluated with ReviewBench before A/B testing, offline changes have consistently pointed in the same direction as what we see later in production. A recent lite-tier experiment provides a concrete example of this broader pattern. We introduced a multi-model ensemble review that combines several independent model runs into a single review rather than relying on a single run. ReviewBench predicted higher precision, recall, and comment volume, along with lower cost per review. To compare offline and production results, we use corresponding online signals. Addressed rate, our online counterpart to precision, is the percentage of CCR comments that an LLM determines prompted a developer to make a corresponding code change, based on the diff, thread, reactions, resolution state, and post-review code. For recall, we measure how much additional human review is still needed. The online A/B test moved in the same direction as ReviewBench predicted: addressed rate (precision) rose 8.0%, recall rose 13.6%, and comment volume rose 61%, while cost per review fell 8.0%, all relative to the production control. Comment volume alone, however, does not capt
What's Changed Check publish limits before uploading, and batch and retry upload requests by @slimbuck in #1066 Full Changelog : v3.5.1...v3.5.2 官方 Release：v3.5.2。请查看 Release notes、依赖和许可证变化。
Added Color workspace. An Edit / Color switch in the top bar. Color swaps the media bin for live scopes of the program monitor (or of the export frame, when one is set) — waveform, RGB parade, vectorscope (75 % targets and a skin-tone line) and histogram , two at a time — and opens the new Color tab beside the Inspector and Mixer. Color grading ( props.grade ) on video, image, SVG and adjustment clips, run on the GPU (WebGL2, with a CPU fallback) before the clip is composited, so preview and export match: lift / gamma / gain / offset wheels with master sliders, exposure and white balance in linear light, a white-balance picker (click something white or grey in the monitor), blacks / shadows / midtones / highlights / whites , log-style contrast with a pivot , soft rolloff into black and white, and saturation . On / off to compare, Reset, and Copy / Paste a grade onto every selected clip. The existing filter sliders and presets still apply on top. Curves in the Color tab: a luma curve (brightness only) and R / G / B curves, plus hue vs hue , hue vs saturation , hue vs luma and saturation vs luma . Click to add a point, drag to move it, double-click to remove it; Pick marks a colour from the monitor on the curve. A hue curve with one point is a band around that hue. Stored in props.grade ( curves , hueHue , hueSat , hueLuma , satLuma ) and set by agents with setGrade . Layers (secondaries) in the Color tab: corrections limited by an HSL qualifier (hue / saturation / luma ranges with softness, invert, and a Pick that selects a colour from the monitor) and / or a mask (ellipse, rectangle or polygon; feather, invert, rotation), dragged on the monitor and keyframable. Matte shows what a layer selects. Each layer has its own wheels, sliders and curves; up to 8 stack in order. Stored in props.grade.layers . Agents edit one layer with the patch op setGradeLayer and check it with fablecut_scopes {matte:{clip, layer}} (coverage, plus the levels and colour of the selected pixels). Agents: the patch op setGrade (merge, reset a key with null , replace , several clips at once; validated, honours locks) and the tool fablecut_scopes , which renders the graded frame exactly as export would and returns luma levels, clipping, mean RGB, saturation and the colour cast as numbers. The keyboard-shortcuts dialog ( ? ) lists the Color workspace gestures (wheels, pickers, curves, masks on the monitor) and the audio volume line. 官方 Release：v1.11.0。请查看 Release notes、依赖和许可证变化。
Text-driven motion editing modifies an existing motion sequence according to a text instruction while preserving the content of the source motion. Existing methods are typically built for a single, fixed skeletal topology, which limits their use in animation pipelines where characters differ in joint count and skeletal hierarchy. We present Topology-Aware Motion Editor (TAME), a flow-matching transformer that edits motions on humanoid skeletons of varying topology. TAME represents motion as per-joint, per-frame tokens and models interactions among joints, across frames, and with the text instruction through skeletal, temporal, and text cross-attention layers. To make the skeletal attention follow each character's hierarchy, TAME replaces full joint attention with Topology-Constrained Skeletal Propagation (TCSP), which restricts attention to one-hop kinematic neighbors in the skeleton's adjacency matrix. We further introduce Edit-Focused Representation Alignment (EFRA), a self-distilled representation alignment strategy that aligns student features with cleaner EMA-teacher features exclusively on edit-relevant joint-time tokens, making edits faithful to the instruction. To make this setting trainable and comparable, we construct TopoMotionFix, a multi-topology extension of MotionFix with seen- and unseen-topology evaluation protocols. TAME outperforms previous methods in edit alignment and source preservation on MotionFix and reliably edits motions on unseen skeletons in TopoMotionFix.
中文 | English ✨ 新增功能 新增实验性 Claude Code Mods 兼容层。目前阶段的主要目的是验证 Claude Code Mods API 功能大致为 DeepSeek Harness 插件的一个子集，而非为用户提供实际的完整兼容性。 @tianyicui 插件管理页新增「让 Agent 创建插件」入口，可保留草稿进入创造模式；发送需求后才开始执行。 @ZiyaZhang 新会话支持预填未发送的提示；恢复草稿或切换工作区时保留文件、目录和会话引用。 @imccyu Markdown 文件预览将 YAML 头部元数据显示为清晰的字段列表。 @turtle2099 Web 支持通过 --public-url 指定显示、打开及提供给模型的对外访问地址，支持带路径前缀的反向代理入口。 @oraluben 新增可选的开发者工具组合包，提供会话原始日志、聊天分组与正文的双向定位，以及内嵌 Host 调试工具；内嵌 DevTools 目前使用英文界面。 @imccyu 🐛 问题修复 目标编辑支持多行文本和 Shift+Enter 换行，并改善目标与排队消息编辑器的中文输入及宽度适配。 @turtle2099 修复目标任务停止、恢复并再次停止后，新消息滞留队列的问题。 @turtle2099 修复部分 Bash、PowerShell 和文件修改记录无法展开输出或差异的问题。 @turtle2099 关闭代码工作视图后仍可选择标准、创造和自定义模式；内置 PTC／极简默认值改为标准，已有会话保持原模式。 @ZiyaZhang 修复启停插件时其他插件的样式被移除、需要刷新页面才能恢复的问题。 @turtle2099 登录请求未收到响应时显示网络检查提示，便于区分连接问题与其他登录失败。 @lsdsjy 桌面端默认使用系统分配的端口，避免 Windows 保留端口范围导致应用无法启动。 @turtle2099 修复侧栏列表重排或重新显示后，会话加载动画不同步的问题。 @turtle2099 启用开发目录监听后，HMR 可刷新包入口和依赖映射配置，并正确重载仍在运行的原入口。 @imccyu 修复新组合包启用后找不到依赖，以及停用或卸载后残留依赖映射的问题；替换已安装包的版本仍需重启。 @imccyu 🎨 体验优化 改善自动化任务详情在窄窗口中的布局，关联会话入口会按可用空间收起为图标。 @turtle2099 组合包详情页显示代码来源和当前版本，便于查看安装来源及重装所需的信息。 @turtle1999 工具调用在准备阶段即可显示已到达的命令说明、文件路径和内容生成进度。 @imccyu 加快大量会话的列表读取，同时继续为其他操作提供执行机会。 @turtle1999 插件安装结果显示实际安装版本；因新版本冷却策略安装较旧版本时，提供说明和精确版本安装方式。 @turtle2099 ⚠️ 其他变更 自动化任务改为 Web 内置能力，提醒工具按模式提供：标准、创造和 PTC 模式可用，极简模式和子代理不可用；旧实验组合包选择会自动清理，已有任务保留。 @Chinesezjc , @turtle1999 子路径插件不再读取独立的 package.json ，显示文本和图标须通过对应子路径导出；插件开发模板同步补齐本地化信息与图标。 @turtle1999 破坏性变更：移除运行时 invariant 插件及各包的 ./invariant 导出；依赖这些诊断入口的扩展和自定义 profile 需要调整。 @turtle2099 输入区统计扩展拆为 activity 和 usage 两个独立入口；覆盖旧 stats 整行的插件需要更新注册 ID。 @turtle1999 New Features Add an experimental Claude Code Mods compatibility layer. At this stage, the main goal is to verify that the Claude Code Mods API's capabilities are broadly a subset of what DeepSeek Harness plugins can do, rather than to offer users complete, practical compatibility. by @tianyicui Add a “Let Agent create a plugin” entry to plugin management. It opens Creator mode while preserving the draft and starts work only after the request is sent. by @ZiyaZhang Support unsent initial prompts for new sessions and preserve file, folder, and session references when restoring drafts or moving them between workspaces. by @imccyu Display YAML frontmatter as a readable field list in Markdown file previews. by @turtle2099 Add --public-url for the Web address shown to users, opened in the browser, and supplied to the model, including reverse-proxy URLs with a path prefix. by @oraluben Add an optional Developer Tools bundle with raw session logs, bidirectional navigation between chat groups and the conversation, and embedded Host diagnostics. The embedded DevTools currently uses an English interface. by @imccyu Bug Fixes Support multiline goal editing with Shift+Enter, and improve IME handling and resizing in goal and queued-message editors. by @turtle2099 Fix new messages remaining queued after stopping, resuming, and stopping a goal-driven task again. by @turtle2099 Fix some Bash, PowerShell, and file-mutation records failing to expand their output or diff. by @turtle2099 Keep Standard, Creator, and custom modes available when coding view is disabled. Built-in PTC or Minimal defaults change to Standard, while existing sessions retain their mode. by @ZiyaZhang Fix toggling a plugin removing styles owned by other plugins until the page is refreshed. by @turtle2099 Show a network-check message when a sign-in request receives no response, distinguishing connection problems from other sign-in failures. by @lsdsjy Let the operating system assign the Desktop port by default, avoiding startup failures caused by reserved port ranges on Windows. by @turtle2099 Keep session loading animations synchronized after sidebar rows move or become visible again. by @turtle2099 With development-directory watching enabled, refresh package-entry and dependency-mapping configuration while continuing to reload the entry that is already running. by @imccyu Fix missing dependencies after enabling a new bundle and stale dependency mappings after disabling or uninstalling bundles. Replacing an installed package version still requires a restart. by @imccyu Improvements Improve Automation task details in narrow windows by collapsing the linked-session action to an icon when space is limited. by @turtle2099 Show a bundle’s code source and current version in its details, making installation and reinstallation information easier to find. by @turtle1999 Show available command descriptions, file paths, and content-generation progress while tool calls are still preparing. by @imccyu Speed up large session listings while periodically allowing other work to run. by @turtle1999 Show the version actually installed and explain how to request an exact version when pnpm’s release-age policy selects an older release. by @turtle2099 Chores Make Automation tasks part of Web, with reminder tools scoped to Standard, Creator, and PTC modes. Minimal mode and subagents do not receive them. Retired experimental-bundle selections are cleaned up while existing tasks are preserved. by @Chinesezjc , @turtle1999 Stop reading separate package.json files for subpath plugins; display text and icons must use the corresponding subpath exports. Plugin-development templates now include localized metadata and icons. by @turtle1999 Breaking change: remove runtime invariant plugins and package ./invariant exports. Extensions and custom profiles using these diagnostic entry points must be updated. by @turtle2099 Split composer statistics into separate activity and usage extension entries. Plugins replacing the former stats row must update their registered IDs. by @turtle1999 Full Changelog: dsh-v0.2.0-rc.2...dsh-v0.2.1-alpha.1 官方 Release：dsh-v0.2.1-alpha.1。请查看 Release notes、依赖和许可证变化。
We present FlowHMR, a framework for recovering physically plausible global 3D human motion from monocular video. Previous learning-based methods typically regress human motion directly from video and train the network with geometric supervision. However, recovering human motion from monocular video is inherently ambiguous in depth, and direct regression tends to collapse toward an averaged solution. Moreover, the recovered motions are not guaranteed to be physically plausible, so physics-based tracking of them often fails. To address these challenges, we formulate video motion capture as a video-conditioned motion generation problem and first pretrain a flow matching model for this task. Given an input video, the pretrained model generates diverse motion candidates, but not all of them are faithful to the video or physically trackable. We therefore post-train the model using Group Relative Policy Optimization (GRPO) with two rewards. A fidelity reward encourages consistency with the input video. A tracking reward favors motions that a physics-based controller can track successfully. Together, these rewards shift the model's output preference, so the post-trained model stays faithful to the input video while producing more physically plausible motion. We further introduce Wild-4K, a large and diverse dataset of about 4K internet videos, for evaluating human motion recovery in the wild. Qualitative and quantitative experiments on Wild-4K show that our method outperforms state-of-the-art methods in overall motion fidelity and achieves a physical tracking success rate of 82.47%, compared with 62.82% for the strongest baseline, GVHMR.
What's Changed Strip upper-case extensions from the default export filename by @Arthur031221 in #1059 Update all npm dependencies by @renovate [bot] in #1060 New Contributors @Arthur031221 made their first contribution in #1059 Full Changelog : v3.4.2...v3.5.0 What's Changed Require LODs when publishing scenes over 40 million splats by @slimbuck in #1064 Update all npm dependencies by @renovate [bot] in #1065 Full Changelog : v3.5.0...v3.5.1 官方 Release：v3.5.1。请查看 Release notes、依赖和许可证变化。
What's Changed Strip upper-case extensions from the default export filename by @Arthur031221 in #1059 Update all npm dependencies by @renovate [bot] in #1060 New Contributors @Arthur031221 made their first contribution in #1059 Full Changelog : v3.4.2...v3.5.0 官方 Release：v3.5.0。请查看 Release notes、依赖和许可证变化。
Editable 3D models of field-grown crops support high-throughput phenotyping and in silico breeding trials, but building them from scanned point clouds requires organ-level segmentation and fitting. Procedural generators can turn an organ-level parameter set into an analysis-suitable 3D model, but obtaining that set requires hours of manual tuning per plant or segmentation models trained on species-specific labels. We present an automated pipeline that reconstructs procedural maize models from raw 3D point clouds without manual tuning or species-specific training data. A multimodal vision-language model (VLM) annotates leaf midlines in rendered orthographic views. Deterministic geometric algorithms back-project the annotations onto the point cloud, merge them into 3D leaves by cross-view consensus, and grow the midlines to full blades on an orientation-weighted surface graph. Measured organ parameters populate a plant descriptor for a Non-Uniform Rational B-Spline (NURBS)-based procedural model generator. Each leaf surface is then refined against its scan points by differentiable NURBS fitting. The pipeline reached a median whole-plant Chamfer distance of 5.4 mm on 100 genotypically diverse field-grown maize plants from the MaizeField3D dataset. The reconstructions were closer to the scans than those of an earlier semi-automated pipeline based on manual annotations. The pipeline recovered 1,017 of 1,023 (99.4%) curated reference leaves at an intersection-over-union of at least 0.5 without using those labels as input. These results show that VLM annotations become usable organ-level measurements when downstream geometric stages can correct them. This makes automated generation of editable 3D plant assets feasible at the scale of modern phenotyping experiments.
AI is changing how developers work and apply their skills. Writing code is still essential, but developers increasingly need to know how to direct AI, evaluate its output, communicate tradeoffs, and make sound technical decisions. The good news? You can start preparing today. Here’s where to focus: Tip #1: Learn to direct AI, not just use it. AI is changing what execution looks like. Increasingly, great execution means defining the problem clearly, providing the right context, evaluating AI-generated code, and deciding what’s ready to ship. As AI agents take on more of the implementation, these skills become even more valuable. For example, imagine you’re asked to add a new authentication flow. A traditional workflow might look like this: Task: Add authentication → Create branch → Write code → Run tests → Open pull request As AI tools become more capable and more deeply integrated into day-to-day workflows, those same skills can help you coordinate multiple AI agents. Your workflow might look more like this: Workspace: Add authentication Agent 1 ✓ Authentication ready for review Agent 2 ✓ Documentation draft ready Agent 3 ✓ Test suite ready Notice what changed. You’re still responsible for the outcome, but you’re spending less time implementing every piece yourself. Instead, you’re defining the work, reviewing outputs, and making the technical decisions that bring everything together. Takeaway: Learn to direct AI agents. Start your first agent session > Tip #2: Don’t trust AI’s first answer AI can generate impressive solutions in seconds, but the first answer isn’t always the best. Your experience writing clean, maintainable code can help you evaluate AI’s output. Ask a second AI model to critique the first model’s work, then use your own judgment to evaluate both responses. Here’s a prompt that shows what that might look like in practice: "Write a SQL query that returns each customer's most recent order." ↓ AI Model #1 ✓ Generates the query ↓ AI Model #2 (Critique) ⚠ Doesn't handle duplicate timestamps ⚠ Missing index recommendation ⚠ May perform poorly on large tables Different AI models have different strengths and blind spots. That’s why GitHub Copilot’s built-in Rubber Duck agent uses a second model to critique plans, code, and tests before you move forward. A second perspective often catches issues the first model misses. Takeaway: Trust AI enough to use it but not enough to skip review. Tip #3: Use AI to solve bigger problems When AI saves implementation time, developers can use that time to tackle broader problems: understanding customer needs, evaluating tradeoffs, designing better systems, and making technical decisions that AI can’t make for them. For example: Issue #4821 Title: Add dark mode AI ✓ Build implementation ✓ Generate tests ✓ Update documentation Developer checklist ☐ Validate customer problem ☐ Review architectural tradeoffs ☐ Check accessibility ☐ Define success metrics ☐ Approve solution As AI takes on more implementation work, the skills that distinguish great engineers become even more important: exercising sound judgment, balancing tradeoffs, and solving the right problems. Takeaway: Let AI handle more of the implementation so you can spend more time building the judgment that helps teams make better decisions. The bottom line As developer workflows evolve, learning to work effectively with AI—and strengthening your own technical judgment—can help you adapt. Put these ideas in practice > The post AI is changing developer work. Here are three skills to strengthen. appeared first on The GitHub Blog .
Added Mixer. A Mixer tab beside the Inspector: one strip per audio track with a fader (console taper, unity at ¾ throw, double-click for 0 dB, or type a level), pan, mute, solo and a meter, plus a master fader with a live LUFS readout. Levels are saved as tracks[].gain / tracks[].pan and master.gain in project.json , written only when they differ from 0. Clip gain ( props.gain , dB): a trim before volume, so volume keyframes and fades keep working on top of it. Channels ( props.channelMode ): stereo, mono (L+R folded to one channel), left, right or swap, per clip. Normalize. In the inspector, pick a target (−14 LUFS streaming, −16 LUFS podcast, −23 LUFS broadcast, −1 dBFS peak) and Apply : the selected clips are measured (ITU-R BS.1770 integrated loudness with EBU R128 gating, or sample peak) and their clip gain set to hit it. Linked stems are measured together and share one gain. MCP: fablecut_normalize_audio does the same from an agent (needs ffmpeg), with the same measurement as the editor. fablecut_patch_project gains a setTrack op for track faders / pan and a master field on setProject ; the compact project view shows a MIX: line when any level is off 0. Audio effects. Effect chains on audio clips (Inspector → Audio effects), tracks and the master (Mixer → FX): EQ (3-band), high-pass, low-pass, compressor, limiter (brickwall, 5 ms lookahead), noise gate, delay, reverb, distortion and stereo width. Reorder, bypass or tweak each effect; preview and export run the same chain. Saved as fx on clips, tracks[].fx and master.fx . Audio presets. Voice: Clean voice, Podcast, Radio, Deep voice (EQ-based), Telephone. Music: Cinematic, Wide, Muffled. A preset writes ordinary effects into the chain, so it can be adjusted afterwards. MCP: fablecut_patch_project gains setFx (a chain or a preset on a clip and its linked stems, a track or the master; validated). The compact view lists chains. The MCPB bundle manifest now lists every tool ( fablecut_encode_profiles was missing). Volume line. Audio clips draw their volume as a line: drag it to change the level (with keyframes, the two around the pointer move together), Ctrl/Cmd-click to add a keyframe, drag a point to move it, Alt-click or double-click to remove one. Edits apply to linked stems too. Fade grips and fade curves. Drag the grip in a clip's top corner to fade in or out. Audio fades take a curve : constant power, constant gain or exponential (inspector → Transition), drawn as the real curve on the clip. Fades without a curve sound exactly as before. Crossfade (Shift+D, or the clip menu). Crossfades the audio cut next to the selection, or the one at the playhead: the overlap is borrowed from spare media on both sides of the cut, so nothing downstream moves; linked picture dissolves in. Overlapping audio that is crossfaded no longer shows the overlap warning. Auto-duck. In the inspector (or the clip menu): the selected music dips by the chosen amount wherever the dialogue tracks have sound, ramping down just before and back up after, with short pauses bridged. Written as a new duck keyframe channel (dB) on top of volume, so the music's own levels are never touched and a re-run replaces the old dips. MCP: fablecut_auto_duck , same detection as the editor. Agents can export. fablecut_export renders the timeline to a file in exports/ with the editor's own Fast export and returns its path. An open editor tab takes the job, or the server starts a headless Chrome / Edge ( FABLECUT_CHROME to choose) and closes it afterwards. Range, encoding profile, waiting, status and cancel are options. REST: POST /api/export/request and GET /api/export/job . MCP: fablecut_patch_project runs the editor's own timeline edits — split , rippleDelete , closeGap , lift , extract , insert , overwrite , rippleTrim , roll , slip , slide and crossfade — with the same linked-stem sync, track targeting and locks as the keyboard shortcuts. setProject takes inPoint / outPoint , and the compact view shows them. Pitch shift effect (±12 semitones, speed unchanged). The Deep voice preset now lowers the pitch (−4 semitones) instead of only EQ-ing the tone. Effect automation. Any effect parameter (except reverb decay) can follow keyframes: a ◆ beside each slider sets or removes a key at the playhead, and the slider then edits the key under the playhead. Clip, track, bus and master effects; preview and export follow the same curve. Saved as keys on the effect. MCP: the setFxKeys patch op, or keys inside a setFx chain. Submix buses. + Bus in the Mixer adds a bus with its own effects, fader, pan, mute and meter; the menu under a track's name routes it into a bus instead of the master. Saved as buses and tracks[].out . MCP: setBus , removeBus , setTrack {out} , and setFx / setFxKeys on a bus. Noise reduction. The inspector's Noise control (Light / Medium / Strong / Off) measures the file's noise floor and renders a cleaned copy with ffmpeg's FFT denoiser; the clip's audio stems switch to it, and Off switches back. MCP: fablecut_denoise ; REST: POST /api/denoise . Needs ffmpeg. Changed The editor's split, ripple, insert, trim and crossfade code moved into edit-ops.js , which the editor and the MCP server both load. The master loudness readout is measured on the finished mix, after master effects and the master fader. Preview and export now mix audio through the same graph: each clip feeds its track, each track the master. Before, the export mix sent every clip straight to the output, so anything set on a track or the master could not reach the exported file. Fixed Titles draw in their own fonts again. Every title style (Anton, Playfair Display, Bebas Neue, Caveat, …) was falling back to the system sans-serif: The 20 fonts in library/fonts were the wrong slice of each Google font (Cyrillic, Vietnamese, …) with no A–Z in them. They are now the Latin files, and the variable ones (Caveat, DM Sans, Inter, Montserrat, Oswald, Playfair Display, Roboto, Rubik, Teko) keep their whole weight range, so weight 100–900 draws real weights instead of a faked bold. Fonts that aren't in the library (Cinzel for the Luxury style, or any Google Font typed by name) never loaded: the check meant to skip fonts already present passed for every name. They load now, and Load says so when Google Fonts has no family by that name. A data directory seeded before this release (the Claude Code plugin keeps one) gets the new font files on the next start; a file you changed or added yourself is never overwritten. Exports wait for every title font, including a font-cut title's whole set, before the first frame, so an opening frame can't come out in the fallback face. The font list no longer shows a library font twice after a quick run of project reloads. 官方 Release：v1.10.0。请查看 Release notes、依赖和许可证变化。
Despite never being supervised on explicit 3D motion, large-scale text-to-video diffusion models synthesize realistic human motion in their generated videos. We ask whether this implicit knowledge can be turned into explicit 3D motion generation, without training a separate motion model. Probing a frozen Wan2.1 reveals that a recoverable motion signal is present in its intermediate states across the entire denoising schedule, not confined to the clean output. Motivated by this, we introduce parasitic co-denoising, a paradigm in which motion is decoded from the host model along its denoising schedule rather than produced by an independent generator. We instantiate it as the Parasitic Motion Decoder (PMD), an efficient flow-matching decoder that shares the host's noise schedule and reads its intermediate features through a $σ$-adaptive multi-layer fusion, leaving the host unmodified. Drawing its coverage from the host rather than from motion data, PMD leads dedicated motion generators on text-motion alignment at a small fraction of their trainable parameters, while producing paired video and motion in a single pass that motion-only baselines cannot match.
Most human motion generation methods encode motion as tokens on a uniform temporal grid, where every token spans the same fixed time window. Human motion, however, is temporally heterogeneous: slowly evolving global trajectories coexist with rapid transient events such as foot contacts and joint impulses. Forcing such multi-scale dynamics onto tokens of identical temporal resolution entangles motion frequencies, leaving slow regions redundant while smoothing out the rapid details that distinguish realistic motion. We propose \textbf{FreqMo}, a scale-adaptive motion representation that decomposes motion into wavelet frequency bands, separating dynamics across temporal scales while preserving temporal localization and exact reconstruction. Unified Frequency Residual Quantization (UFRQ) then encodes all bands within a single shared codebook, compressing the token sequence threefold and enabling stable single-stage generation. Experiments show FreqMo attains SOTA fidelity with substantially improved high-frequency preservation, and the same decomposition transfers to continuous diffusion backbones.
Recent text-to-motion models have improved motion quality and instruction following, yet many-step denoising and large model components make deployment slow and memory-intensive. We present Terminal-Amplification-Controlled Distillation (TACD), an on-policy approach for training efficient motion generators from text prompts and pretrained teachers, without real-motion training data. Building on segmented on-policy flow distillation, we supervise clean-motion predictions along student-generated trajectories. We identify a failure mode in which velocity matching on a fixed supervision grid repeatedly overweights errors near the denoising endpoint, degrading few-step generation. TACD ties the latest teacher query to the student's step size, bounding the effective loss weights in clean-motion space without changing inference. Experiments on HumanML3D and KIT-ML demonstrate improved few-step generation, including a 58% reduction in eight-step HY-Motion student FID relative to distillation without this bound. For diffusion teachers, the endpoint-matching form of TACD yields four-step students with lower FID and matched or improved text-motion retrieval relative to their 50-step teachers on HumanML3D. On HY-Motion and Kimodo, eight-step students with compact components achieve 7.7-11.9x end-to-end speedups and reduce peak GPU memory by 3.8-6.7x relative to their teachers. Project page: https://vkgo.github.io/TACD/
3D editing methods are usually tested on a single edit, yet an asset is built through a long sequence of revisions, each of which must implement the requested change while leaving everything else unchanged. We introduce EditHero, to our knowledge the first benchmark for long-horizon, part-level 3D editing, with natural-language instructions and target images for both geometry and texture. A deterministic assembly engine produces the exact target after every edit, and every sequence is reviewed by hand. We use EditHero to compare 2 opposite approaches to 3D editing. Non-agentic methods operate top down, regenerating the object from a learned 3D representation and inferring what to keep. In contrast, LLM/VLM agents operate bottom up, editing through code that inspects the mesh and rewrites only the parts required by instructions. The non-agentic methods often miss the requested change and disturb regions that should stay fixed. Most LLMs follow instructions more closely, and all of them preserve the unedited parts better, but each of their edits takes minutes. We will release the engine and the edit sequences to support research on reliable iterative 3D editing.
We present LiteReality-Agent, an agentic system for reconstructing real indoor environments as realistic, articulated, and simulation-ready 3D scenes from RGB-D scans. At its core, LiteReality-Agent formulates 3D reconstruction as a coding problem, in which a coding agent gathers evidence using specialised tools and iteratively edits a Python script, Room.py, which can be executed to produce a 3D digital twin of the room. With this formulation, we develop a robust observe-edit-verify harness that supports evidence gathering, measurement, verification, layout optimisation, simulation readiness, and quality control throughout the reconstruction process. LiteReality-Agent produces high-quality reconstructions suitable for simulation and downstream embodied AI tasks. Furthermore, as agent capabilities continue to improve rapidly, the system introduced by LiteReality-Agent remains a strong orchestration framework for future agents: it equips them with specialised tools, structured workflows, and robust verification mechanisms that substantially improve reconstruction quality and reliability. We demonstrate that LiteReality-Agent produces reconstructions that are more geometrically accurate, visually realistic, and simulation-compatible than those generated by recent frontier models, such as Astra and Fable. We therefore view LiteReality-Agent as a practical and important building block for robust real-to-sim systems. Both the source code and the data-capture application are publicly available. Code:https://github.com/LiteReality/LiteReality-Agent/