Ph.D. student Ziqi Zhang has built a career blending machine learning with single-cell biology. His work helps scientists study cellular mechanisms that advance disease research and drug development.

Though decorated with awards and appearances in leading journals, Zhang will achieve his greatest accomplishment tonight at McCamish Pavilion. He will join the Class of 2025 in walking across the stage, receiving diplomas, and graduating from Georgia Tech.

Before he “gets out” of Georgia Tech, we interviewed Zhang to learn more about his Ph.D. journey and where his degree will take him next. 

Graduate: Ziqi Zhang

Research Interests: Machine learning, foundational models, cellular mechanisms, single-cell gene sequencing, gene regulatory networks

Education: Ph.D. in Computational Science and Engineering

Faculty Advisor: School of CSE J.Z. Liang Early-Career Associate Professor Xiuwei Zhang

What persuaded you to study at Georgia Tech? 

I chose Georgia Tech because it is one of the top engineering institutions in the United States, known for its strength in machine learning and data science. The university offers exceptional research resources and the opportunity to work with leading scholars in my field. Georgia Tech also has very good research infrastructure. The Coda Building is one of the most well-designed and productive research environments I have experienced. Having access to such a space has been a genuine privilege.

How has working on your CSE degree helped you so far in your career?

Working toward my CSE degree has been instrumental in my career development. As an interdisciplinary program, CSE has equipped me with strong computational skills while also deepening my understanding of key application domains. This breadth of training has opened more opportunities during my job and internship searches. In addition, CSE community events, such as HotCSE, the weekly coffee hour, and faculty recruiting activities, have helped me strengthen my scientific communication skills, which are essential for my long-term career growth.

What research project from Georgia Tech are you most proud of?

My favorite research project was scMoMaT, a matrix tri-factorization algorithm for single-cell data integration. I invested a significant amount of time and effort into this work, iterating on the model many times. I’m very proud that it ultimately evolved into a clean, robust, and elegant algorithm.

What advice would you give someone interested in graduate school?

It is important to find an advisor who is supportive and genuinely invested in your career development. A Ph.D. is not an easy journey, and you will inevitably encounter challenges along the way. Having an advisor who can provide thoughtful guidance and dedicated mentorship is one of the most crucial factors in helping you navigate those difficulties.

What is your most favorite memory from Georgia Tech?

CSE’s new student campus visit day every year was one of my favorite times of the year. It was always fun to meet new people, have good food, and enjoy the beautiful view from the Coda rooftop.

What are your plans after graduation?

I plan to keep working in academia after graduation. I’m on the job hunt, currently applying for positions and preparing for interviews.

Proteins, including antibodies, hemoglobin, and insulin, power nearly every vital aspect of life. Breakthroughs in protein research are producing vaccines, resilient crops, bioenergy sources, and other innovative technologies.

Despite their importance, most of what scientists know about proteins only comes from a small sample size. This stands in the way of fully understanding how most proteins work and unlocking their full potential.

Georgia Tech’s Yunan Luo believes artificial intelligence (AI) could fill this knowledge gap. The National Science Foundation agrees. Luo is the recipient of an NSF Faculty Early Career Development (CAREER) award. 

“So much of biology depends on knowing what proteins do, but decades of research have concentrated on a relatively small set of well-studied proteins. This imbalance in scientific attention leads to a distorted view of the biological landscape that quietly shapes our data and our algorithms,” Luo said.

“My group’s goal is to build machine learning (ML) models that actively close this gap by generating trustworthy function predictions for the many proteins that remain understudied.”

[Related: Yunan Luo to use AI for Protein Design and Discovery with Support of $1.8 Million NIH Grant]

In his proposal to NSF, Luo coined this rich-get-richer effect “annotation inequality.” 

One problem of annotation inequality is that it slows progress in disease prognosis, drug discovery, and other critical biomedical areas. It is challenging to innovate the few proteins that scientists already know so much about. 

A cascading effect of annotation inequality is that it diminishes the effectiveness of studying proteins with AI.  

AI methods learn from existing experimental data. Datasets skewed toward well-known proteins propagate and become entrenched in models. Over time, this makes it harder for computers to research understudied proteins. 

“Protein annotation inequality creates an effect analogous to a vast library where 95% of patrons only read the top 5% popular books, leaving the rest of the collection to gather dust,” Luo said.

“This has resulted in knowledge disparities across proteins in current literature and databases, biasing our understanding of protein functions.”

The NSF CAREER award will fund Luo with over $770,000 for the next five years to tackle head-on the problem of protein annotation inequality.

Luo will use the grant to build an accurate, unbiased protein function prediction framework at scale. His project aims to:

• Reveal how annotation inequality affects protein function prediction systems
• Create ML techniques suited for biological data, which is often noisy, incomplete, and imbalanced  
• Integrate data and ML models into a scalable framework to accelerate discoveries involving understudied proteins

More enduring than the ML framework, Luo will leverage the NSF award to support educational and outreach programs. His goal is to groom the next generation of researchers to study other challenges in computational biology, not just the annotation inequality problem.

Luo teaches graduate and undergraduate courses focused on computational biology and ML. Problems and methods developed through the CAREER project can be used as course material in his classes.

Luo also championed collaboration with Georgia Tech’s Center for Education Integrating Science, Mathematics, and Computing (CEISMC) in his proposal. 

Through this partnership, local high school teachers and students would gain access to his data and models. This promotes deeper learning of biology and data science through hands-on experience with real-world tools.  

Luo sees reaching students and the community as a way of paying forward the support he received from Georgia Tech colleagues. 

“I am incredibly grateful for this recognition from the NSF,” said Luo, an assistant professor in the School of Computational Science and Engineering (CSE). 

“This would not have been possible without my students and collaborators, whose hard work laid the groundwork for this proposal.”

Luo praised CSE faculty members B. Aditya Prakash, Xiuwei Zhang, and Chao Zhang for their guidance. All three study machine learning and computational bioscience, two of CSE’s five core research areas. 

Luo also thanked Haesun Park for her support and recommendation for the CAREER award. Park is a Regents’ Professor and the chair of the School of CSE.

The Institute for Matter and Systems (IMS) hosted the inaugural Boundaries and Breakthroughs panel on Nov. 11, setting the stage for a new era of interdisciplinary dialogue at Georgia Tech. The event, held in the Marcus Nanotechnology building, brought together experts in electrical engineering, computer architecture, and computer systems design to tackle one of today’s pressing challenges: artificial intelligence (AI) scalability and sustainable high-performance computing.

As one of Georgia Tech’s 11 interdisciplinary research institutes, IMS is designed to break down silos between traditional academic units. By operating core user facilities and fostering collaborative research, IMS creates a unique ecosystem where device-level innovation meets systems-level design. This event personified that mission by connecting researchers who typically work on different ends of the stack.

“We’re looking for opportunities to bring people together to have discussions that are both informative and potentially create a little bit of friction in the best possible way around trending topics in science and engineering,” said Mike Filler, IMS deputy director, during opening remarks.

The panel was moderated by Divya Mahajan, assistant professor in the School of Electrical and Computer Engineering, and featured Moinuddin Qureshi, professor of computer science; Anand Iyer, assistant professor of computer science; and Asif Khan, associate professor in electrical and computer engineering. 

The discussion explored the dynamics between compute abundance and energy constraints. As AI models scale up, power consumption has become a societal issue, driving up energy demands and even influencing political conversations. The panelists agreed that the bottleneck isn’t compute — a computer’s ability to process and execute tasks — but data movement. Moving data uses 100 to 1,000 times more energy than computation, making memory systems the critical frontier.

The conversation highlighted how breakthroughs in compute must occur at every layer — from individual devices to full computer systems. At the device level, Khan mentioned emerging memory technologies and “beyond CMOS” approaches such as embedding compute within memory and exploring bio-inspired architectures.

From a computer architecture level, Qureshi advocated rethinking interfaces and creating designs optimized for the future of computing. AI needs regular patterns to work optimally, and current patterns are not set up for that.

“If you want efficiency, design systems that make sense for AI,” Qureshi said. “Develop new interfaces, develop new modules, architectures, and organization that make for a specific pattern.”

At the systems level, Iyer stressed practical strategies like near-memory compute and energy-aware scheduling while acknowledging the need for co-design between hardware and software.

“Now in terms of brains or bio-inspired computing, my conjecture is that there is currently no hardware that is capable of doing it,” Khan said. He also noted that right now, there is no computer or algorithm that has the scale of computing comparable to human brain power.

The panelists didn’t shy away from provocative ideas — such as whether graphic processing units are the final solution for AI and whether matrix multiplication alone can lead to artificial general intelligence. While opinions varied, all agreed that organizations like IMS are key to bringing together diverse expertise to tackle these questions collaboratively.

The Boundaries and Breakthroughs series continues in January with a panel on bioelectronics and medical technologies, reinforcing IMS’s commitment to fostering dialogue that spans the full spectrum of innovation.

Viral videos abound with humanoid robots performing amazing feats of acrobatics and dance but finding videos of a humanoid robot performing a common household task or traversing a new multi-terrain environment easily, and without human control, are much rarer. This is because training humanoid robots to perform these seemingly simple functions involves the need for simulation training data that lack the complex dynamics and degrees of freedom of motion that are inherent in humanoid robots. 

To achieve better training outcomes with faster deployment results, Fukang Liu and Feiyang Wu, graduate students under Professor Ye Zhao from the Woodruff School of Mechanical Engineering and faculty member of the Institute for Robotics and Intelligent Machines, have published a duo of papers in IEEE Robotics and Automation Letters. This is a collaborative work with three other IRIM affiliated faculties, Profs. Danfei Xu, Yue Chen, and Sehoon Ha, as well as Prof. Anqi Wu from School of Computational Science and Engineering.

To develop more reliable motion learning for humanoid robots and enable humanoid robots to perform complex whole-body movements in the real world, Fukang led a team and developed Opt2Skill, a hybrid robot learning framework that combines model-based trajectory optimization with reinforcement learning.  Their framework integrates dynamics and contacts into the trajectory planning process and generates high-quality, dynamically feasible datasets, which result in more reliable motion learning for humanoid robots and improved position tracking and task success rates. This approach shows a promising way to augment the performance and generalization of humanoid RL policies using dynamically feasible motion datasets. Incorporating torque data also improved motion stability and force tracking in contact-rich scenarios, demonstrating that torque information plays a key role in learning physically consistent and contact-rich humanoid behaviors.

While other datasets, such as inverse kinematics or human demonstrations, are valuable, they don’t always capture the dynamics needed for reliable whole-body humanoid control.” said by Fukang Liu. “With our Opt2Skill framework, we combine trajectory optimization with reinforcement learning to generate and leverage high-quality, dynamically feasible motion data. This integrated approach gives robots a richer and more physically grounded training process, enabling them to learn these complex tasks more reliably and safely for real-world deployment. - Fukang Liu

In another line of humanoid research, Feiyang established a one-stage training framework that allows humanoid robots to learn locomotion more efficiently and with greater environmental adaptability. Their framework, Learn-to-Teach (L2T), unlike traditional two-stage “teacher-student” approaches, which first train an expert in simulation and then retrain a limited-perception student, teaches both simultaneously, sharing knowledge and experiences in real time. The result of this two-way training is a 50% reduction in training data and time, while maintaining or surpassing state-of-the-art performance in humanoid locomotion. The lightweight policy learned through this process enables the lab’s humanoid robot to traverse more than a dozen real-world terrains—grass, gravel, sand, stairs, and slopes—without retraining or depth sensors.

By training an expert and a deployable controller together, we can turn rich simulation feedback into a lightweight policy that runs on real hardware, letting our humanoid adapt to uneven, unstructured terrain with far less data and hand-tuning than traditional methods. - Feiyang Wu

By the application of these training processes, the team hopes to speed the development of deployable humanoid robots for home use, manufacturing, defense, and search and rescue assistance in dangerous environments. These methods also support advances in embodied intelligence, enabling robots to learn richer, more context-aware behaviors.Additionally, the training data process can be applied to research to improve the functionality and adaptability of human assistive devices for medical and therapeutic uses.

As humanoid robots move from controlled labs into messy, unpredictable real-world environments, the key is developing embodied intelligence—the ability for robots to sense, adapt, and act through their physical bodies,” said Professor Ye Zhao. “The innovations from our students push us closer to robots that can learn robust skills, navigate diverse terrains, and ultimately operate safely and reliably alongside people. - Prof. Ye Zhao

Author - Christa M. Ernst

Citations

Liu F, Gu Z, Cai Y, Zhou Z, Jung H, Jang J, Zhao S, Ha S, Chen Y, Xu D, Zhao Y. Opt2skill: Imitating dynamically-feasible whole-body trajectories for versatile humanoid loco-manipulation. IEEE Robotics and Automation Letters. 2025 Oct 13.

Wu F, Nal X, Jang J, Zhu W, Gu Z, Wu A, Zhao Y. Learn to teach: Sample-efficient privileged learning for humanoid locomotion over real-world uneven terrain. IEEE Robotics and Automation Letters. 2025 Jul 23.
 

 

Research Communications Program Manager

 

Klaus Advance Computing Building 1120E | 266 Ferst Drive | Atlanta GA | 30332

The Georgia Tech Library is proud to show a new piece from Hyojin Kwon and Nix Liu Xin on the Media Bridge, Synthetic Ecologies: AI-Translated Matter in Architectural Media.

The four-minute piece, which went live in November, is playing every hour at the ten-minute mark on Media Bridge, located between Price Gilbert and Crosland Tower.

 

About Synthetic Ecologies: AI-Translated Matter in Architectural Media

Rather than accelerating images, Synthetic Ecologies asks “how matter thinks.” New Materialism treats matter as an active partner; design emerges where human intention intra-acts with things, datasets, models, and light. Authorship becomes orchestration, and every dataset a canon—a politics of selection. Method, therefore, is ethics.

The project authors a white model first, fixing space, camera, light, and motion as temporal logic. AI then enters as a translator, moving on rails of depth/normal/segmentation with flow consistency. The surface reveals signs of behavior—gloss, scattering, porosity, accretion—rather than mere style. We publish provenance—sources and biases, node graphs and parameters, timelines and versions—so choices are traceable and contestable, transferable to learning, practice, and public decision.

From plastic toward a synthetic ecology, the work declares architecture thinking with media. AI is not authority but a transparent amplifier; authorship is not surrendered.

Spanning the Library, the Media Bridge is a civic threshold of study and routine, a common ceiling shared day and night. Installed overhead, the flow of plastics becomes a sky that prompts daily audiences to reflect on circulation, responsibility, and bias. Open provenance turns the piece into a public manifesto, linking campus AI literacy and circularity agendas to civic practice. The work does more than show images; it proposes a public curriculum where data and decisions, materials and culture, negotiate in view of the community.

 

Artist Bios

Hyojin Kwon is an Assistant Professor in the School of Architecture at the Georgia Institute of Technology and cofounder of Pre- and Post-, a research-driven design practice based in Atlanta and Boston. Her work explores how digital media — including animation, simulation, and AI-assisted image workflows — can function as both generative and critical tools within architectural design. Situated within a post digital framework, her recent projects investigate material agency and synthetic ecologies, often translating computational processes into civic installations and experimental representations. Her work has been exhibited internationally, including at the Museum of Brisbane, Tokyo Designers Week, Seoul Foundation for Arts and Culture, and Atlanta Contemporary, and supported by institutions such as MacDowell, Art Omi, and Autodesk.

Nix Liu Xin is a spatial computing artist and director, Harvard graduate, and the founder & CEO of OI (Onceness Intelligence), an experiential-AI startup. He envisions a future where 4D–AI interfaces enable everyone to record life moments, relive memories, and design immersive spatial experiences through intelligent 4D media.

He was named to the AACYF 30 Under 30 and has received accolades such as the Harvard Design Studies Domain Award, the CGarchitect 3D Awards, and the MIT AI Film Hack Award for Best Picture. 

Additionally, Nix was featured as an artist at the Lianzhou Photography Biennale. He also co-founded the HarvardXR Conference and the Creative.Tech Community.

“How will AI kill Creature?”
 

That was the question posed to Scheller College of Business Evening MBA students Katie Bowen (’25), Ellie Cobb (’26), and Christopher Jones (’26) in a marketing practicum course that paired them with Creature, a brand, product, and marketing transformation studio.
 

For 10 weeks, the students worked as consultants in a project that challenged them to rethink the role of artificial intelligence in creative industries. Course instructor Jarrett Oakley, director of Marketing at TOTO USA, guided the student project as they developed strategies to help Creature navigate the evolving landscape of AI-driven marketing.
 

Business School Meets Real Business
 

“Nothing accelerates the value of a business school education like applying it in real time to real businesses,” Oakley said. “This course mirrored a consulting engagement, turning classroom learning into actionable expertise through direct collaboration with local firms. It was designed to spark creative thinking, build confidence, and bridge theory with practice.”
 

What began as a traditional strategic analysis quickly evolved into a forward-looking exploration of AI’s impact on branding, user experience, and performance creative. “Our team realized early on that AI wasn’t a threat but a powerful tool,” the students shared. “We found that AI’s real impact lies not in replacing creativity, but in reshaping expectations, accelerating timelines, and redefining performance standards. It also gives forward-thinking agencies like Creature the opportunity to guide clients still catching up to the AI curve.”
 

Creature’s founders, Margaret Strickland and Matt Berberian, welcomed the collaboration. “We solve creative challenges across brand, product, and performance,” said Strickland. “AI is transforming each of these areas. The students helped us see how to stay ahead of the curve.”
 

Students applied frameworks like SWOT, Porter’s Five Forces, and the G-STIC model to diagnose challenges and develop actionable strategies. Weekly meetings with Creature allowed for iterative feedback and refinement.
 

One of the team’s most surprising insights came from primary research: many agencies hesitate to disclose their use of AI, fearing clients will demand lower prices. “We recommended Creature define and share their AI philosophy,” said the students. “Clients want transparency and innovation, and they’ll choose partners who embrace AI, not hide from it.”
 

Creature took the advice to heart. Since the project concluded, the firm has launched a new AI consulting offering, SNSE by Creature, and implemented automation across operations, resulting in a 21% boost in efficiency. They’ve also adopted an AI manifesto to guide future initiatives.
 

A Transformative Student Experience
 

Katie Bowen, Evening MBA '25
“This project let us apply MBA concepts to a real-world business challenge. We dove into Creature’s business and tailored our analysis to their needs. It pushed us to think critically about how companies stay competitive when AI tools are widely accessible. Using strategy, innovation, and marketing frameworks, we bridged theory and practice to deliver forward-looking recommendations.”
 

Ellie Cobb, Evening MBA ‘26
“This project strengthened my ability to use AI effectively in both personal and professional contexts—not just knowing how to use it, but when not to. Exploring such a fast-evolving topic made me more agile and open-minded, ready to follow where research and emerging trends lead.”
 

Christopher Jones, Evening MBA ‘26
“The Marketing Practicum with Creature was an eye-opening experience that deepened my understanding of AI’s impact on business. It sharpened my critical thinking as I navigated conflicting information about AI, and gave me practical insight into business strategy, from integrating new technology to managing innovation and diversifying product offerings.”
 

Education With Impact
 

Oakley believes the practicum will have lasting impact. “These students now understand how traditional marketing strategy integrates with emerging AI capabilities. They’re ready to lead in a rapidly evolving industry.”
 

As AI continues to reshape marketing, partnerships like the one between Scheller and Creature demonstrate the power of collaboration, innovation, and education in preparing future leaders for whatever comes next.

311 chatbots make it easier for people to report issues to their local government without long wait times on the phone. However, a new study finds that the technology might inhibit civic engagement.

311 systems allow residents to report potholes, broken fire hydrants, and other municipal issues. In recent years, the use of artificial intelligence (AI) to provide 311 services to community residents has boomed across city and state governments. This includes an artificial virtual assistant (AVA) developed by third-party vendors for the City of Atlanta in 2023.

Through survey data, researchers from Tech’s School of Interactive Computing found that many residents are generally positive about 311 chatbots. In addition to eliminating long wait times over the phone, they also offer residents quick answers to permit applications, waste collection, and other frequently asked questions.

However, the study, which was conducted in Atlanta, indicates that 311 chatbots could be causing residents to feel isolated from public officials and less aware of what’s happening in their community.

Jieyu Zhou, a Ph.D. student in the School of IC, said it doesn’t have to be that way.

Uniting Communities

Zhou and her advisor, Assistant Professor Christopher MacLellan, published a paper at the 2025 ACM Designing Interactive Systems (DIS) Conference that focuses on improving public service chatbot design and amplifying their civic impact. They collaborated with Professor Carl DiSalvo, Associate Professor Lynn Dombrowski, and graduate students Rui Shen and Yue You.

Zhou said 311 chatbots have the potential to be agents that drive community organization and improve quality of life.

“Current chatbots risk isolating users in their own experience,” Zhou said. “In the 311 system, people tend to report their own individual issues but lose a sense of what is happening in their broader community. 

“People are very positive about these tools, but I think there’s an opportunity as we envision what civic chatbots could be. It’s important for us to emphasize that social element — engaging people within the community and connecting them with government representatives, community organizers, and other community members.”

Zhou and MacLellan said 311 chatbots can leave users wondering if others in their communities share their concerns.

“If people are at a town hall meeting, they can get a sense of whether the problems they are experiencing are shared by others,” Zhou said. “We can’t do that with a chatbot. It’s like an isolated room, and we’re trying to open the doors and the windows.”

Adding a Human Touch

In their paper, the researchers note that one of the biggest criticisms of 311 chatbots is they can’t replace interpersonal interaction.

Unlike chatbots, people working in local government offices are likely to:

• Have direct knowledge of issues
• Provide appropriate referrals
• Empathize with the resident’s concerns

MacLellan said residents are likely to grow frustrated with a chatbot when reporting issues that require this level of contextual knowledge.

One person in the researchers’ survey noted that the chatbot they used didn’t understand that their report was about a sidewalk issue, not a street issue.

“Explaining such a situation to a human representative is straightforward,” MacLellan said. “However, when the issue being raised does not fall within any of the categories the chatbot is built to address, it often misinterprets the query and offers information that isn’t helpful.”

The researchers offer some design suggestions that can help chatbots foster community engagement and improve community well-being:

• Escalation. Regarding the sidewalk report, the chatbot did not offer a way to escalate the query to a human who could resolve it. Zhou said that this is a feature that chatbots should have but often lack.
• Transparency. Chatbots could provide details about recent and frequently reported community issues. They should inform users early in the call process about known problems to help avoid an overload of user complaints.
• Education. Chatbots can keep users updated about what’s happening in their communities.
• Collective action. Chatbots can help communities organize and gather ideas to address challenges and solve problems.

“Government agencies may focus mainly on fixing individual issues,” Zhou said, “But recognizing community-level patterns can inspire collective creativity. For example, one participant suggested that if many people report a broken swing at a playground, it could spark an initiative to design a new playground together—going far beyond just fixing it.”

These are just a few examples of things, the researchers argue, that 311 services were originally designed to achieve.

“Communities were already collaborating on identifying and reporting issues,” Zhou said. “These chatbots should reflect the original intentions and collaboration practices of the communities they serve.

“Our research suggests we can increase the positive impact of civic chatbots by including social aspects within the design of the system, connecting people, and building a community view.”

One of the top conferences for AI and computer games is recognizing a School of Interactive Computing professor with its first-ever test-of-time award.

At its event this week in Alberta, Canada, the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE) is honoring Professor Mark Riedl. The award also honors University of Utah Professor and Division of Games Chair Michael Young, Riedl’s Ph.D. advisor.

Riedl studied under Young at North Carolina State University.

Their 2005 paper, From Linear Story Generation to Branching Story Graphs, highlighted the challenges of using AI to create interactive gaming narratives in which user actions influence the story’s progression. 

In 2005, computer game systems that supported linear, non-branching games were widely used. Riedl introduced an innovative mathematical formula for interactive stories ranging from choose-your-own-adventure novels to modern computer games.

“We didn’t use the term ‘generative AI’ back then, but I was working on AI for the generation of creative artifacts,” Riedl said. “This was before we had practical deep learning or large language models.

“One of the reasons this paper is still relevant 20 years later is that it didn’t just present a technology, it attempted to provide a framework for solving a grand challenge in AI.”

That challenge is still ongoing, Riedl said. Game designers continue to struggle with balancing story coherence against the amount of narrative control afforded to users.

“When users exercise a high degree of control within the environment, it is likely that their actions will change the state of the world in ways that may interfere with the causal dependencies between actions as intended within a storyline,” Riedl and Young wrote in the paper.

“Narrative mediation makes linear narratives interactive. The question is: Is the expressive power of narrative mediation at least as powerful as the story graph representation?”

AIIDE is being held this week at the University of Alberta in Edmonton, Alberta. Riedl will receive the award on Wednesday.

A new deep learning architectural framework could boost the development and deployment efficiency of autonomous vehicles and humanoid robots. The framework will lower training costs and reduce the amount of real-world data needed for training.

World foundation models (WFMs) enable physical AI systems to learn and operate within synthetic worlds created by generative artificial intelligence (genAI). For example, these models use predictive capabilities to generate up to 30 seconds of video that accurately reflects the real world.

The new framework, developed by a Georgia Tech researcher, enhances the processing speed of the neural networks that simulate these real-world environments from text, images, or video inputs.

The neural networks that make up the architectures of large language models like ChatGPT and visual models like Sora process contextual information using the “attention mechanism.”

Attention refers to a model’s ability to focus on the most relevant parts of input.

The Neighborhood Attention Extension (NATTEN) allows models that require GPUs or high-performance computing systems to process information and generate outputs more efficiently.

Processing speeds can increase by up to 2.6 times, said Ali Hassani, a Ph.D. student in the School of Interactive Computing and the creator of NATTEN. Hassani is advised by Associate Professor Humphrey Shi.

Hassani is also a research scientist at Nvidia, where he introduced NATTEN to Cosmos — a family of WFMs the company uses to train robots, autonomous vehicles, and other physical AI applications.

“You can map just about anything from a prompt or an image or any combination of frames from an existing video to predict future videos,” Hassani said. “Instead of generating words with an LLM, you’re generating a world.

“Unlike LLMs that generate a single token at a time, these models are compute-heavy. They generate many images — often hundreds of frames at a time — so the models put a lot of work on the GPU. NATTEN lets us decrease some of that work and proportionately accelerate the model.”