公式動画ピックアップ
AAPL
ADBE
ADSK
AIG
AMGN
AMZN
BABA
BAC
BL
BOX
C
CHGG
CLDR
COKE
COUP
CRM
CROX
DDOG
DELL
DIS
DOCU
DOMO
ESTC
F
FIVN
GILD
GRUB
GS
GSK
H
HD
HON
HPE
HSBC
IBM
INST
INTC
INTU
IRBT
JCOM
JNJ
JPM
LLY
LMT
M
MA
MCD
MDB
MGM
MMM
MSFT
MSI
NCR
NEM
NEWR
NFLX
NKE
NOW
NTNX
NVDA
NYT
OKTA
ORCL
PD
PG
PLAN
PS
RHT
RNG
SAP
SBUX
SHOP
SMAR
SPLK
SQ
TDOC
TEAM
TSLA
TWOU
TWTR
TXN
UA
UAL
UL
UTX
V
VEEV
VZ
WDAY
WFC
WK
WMT
WORK
YELP
ZEN
ZM
ZS
ZUO
公式動画&関連する動画 [WebMMU: Multimodal and Multilingual Evaluation of Agent Reasoning on Web]
Welcome to the AI research bites. This series of short and informative talks showcases cutting-edge research work from ServiceNow AI Research team. The AI Research Bites are open to all, especially those interested in keeping up with the fast-paced AI research community.
Modern web agents can read, but few can see holistically. Despite rapid progress in multimodal LLMs, today's models falter when asked to visually ground UI elements, reason over DOM structures, or edit complex layouts across diverse languages and domains. WebMMU is our attempt to course-correct: a benchmark born from the belief that data, not just models, is the real bottleneck. With real-world website screenshots and three challenging tasks: VQA, sketch-to-UI, and code editing, WebMMU stresses the visual, structural, and multilingual reasoning skills agents must master to operate robustly on the real web. During this talk, Sai Rajeswar Mudumba will dive into what current models get wrong, how our dataset is designed to test (not trick) them, and why better evals are essential for building general-purpose digital agents that can truly see, reason, and build.
Paper: https://webmmu-paper.github.io/assets/WebMMU_2025.pdf
Dataset: https://huggingface.co/collections/mair-lab/webmmu-686777551dc5d822264e36f2
Website: https://webmmu-paper.github.io/
ServiceNow AI Research team: https://www.servicenow.com/research/
250
4