Advancing Curiosity Award to Support Interdisciplinary Research on Tackling Misinformation

Sandlin, Anu  |  May 29, 2019

News Image: 
Advancing Curiosity Award
artificial intelligence
online misinformation
fake news
Micron Foundation
Texas iSchool

The University of Texas at Austin School of Information Associate Professors Matthew Lease and Kenneth R. Fleischmann have been awarded a $150,000 grant from the Micron Foundation for a two-year project, “Tackling Misinformation through Socially-Responsible AI". In addition to Lease and Fleischmann, the project team includes Associate Professor Samuel Baker (English), Associate Professor Natalie Stroud (Communication Studies), and Professor Sharon Strover (Radio-TV-Film).

Socially-responsible artificial intelligence (AI) involves designing AI technologies to create positive societal impacts. “Emerging AI technologies create tremendous potential for both good or harm,” wrote the research team. “We cannot simply assume benign use, nor can we develop AI technologies in a vacuum ignoring their societal implications.” However, how does an abstract goal like socially-responsible AI get implemented in practice to solve real societal problems? “We argue that grounding the pursuit of responsible AI technology in the context of a real societal challenge is critical to achieving progress on responsible AI,” the team writes, “because it forces us to translate abstract research questions into real, practical problems to be solved.”

Grounding the pursuit of responsible AI technology in the context of a real societal challenge is critical to achieving progress on responsible AI.

In particular, the team will pursue socially-responsible AI to tackle the contemporary challenge of combatting misinformation online. While recent AI research has sought to develop automatic AI systems to predict whether online news stories are real or fake, “why should anyone trust a black-box AI model telling them what to believe”, the team asks, “when many people distrust even well-known news outlets and fact-checking organizations? How do people decide ― if it is a conscious and rational choice ― to believe or even circulate what is actually misinformation? How can AI systems be designed for effective human interaction to help people better decide for themselves what to believe?”

The project team brings diverse, relevant expertise and prior work experience on socially-responsible AI and online misinformation. In fact, team members are already collaborating as part of the campus-wide Good Systems Initiative, a UT Bridging Barriers Grand Challenge to design the future of socially-responsible AI. Good Systems has become a major catalyst at UT in promoting projects on socially-responsible AI, and Micron Foundation support for this project will enable the research team to tackle the specific challenges of designing socially-responsible AI to combat misinformation.

“Our project will develop real use cases, interface designs, prototype applications, and user-centered evaluations,” said Lease. “By grounding AI research in the context of specific social problems, designers can directly consider and confront the societal context of use as AI models are conceived and refined.”The team members explain that this will aid the discovery of new insights into how good AI systems can be developed in general to maximize societal benefit.

The Micron Foundation sought proposals from multi-disciplinary research groups and non-profit organizations investigating how artificial intelligence, machine leaning, and deep learning can improve life while also addressing ethical issues, security, and privacy. Established in 1999 as a private, nonprofit organization with a gift from Micron Technology, the Micron Foundation’s grants, programs, and volunteers focus on promoting science and engineering education and addressing basic human needs.

Microsoft Research Partners with UT Austin, Texas iSchool for Microsoft Ability Initiative

Sandlin, Anu  |  Mar 29, 2019

News Image: 
Image Caption: 
From left to right: Danna Gurari, University of Texas; Ed Cutrell, Microsoft Research; Roy Zimmermann, Microsoft Research; Meredith Ringel Morris, Microsoft Research; Ken Fleischmann, University of Texas; Neel Joshi, Microsoft Research
Microsoft Ability Initiative
Texas iSchool
Microsoft Research
Danna Gurari
Ken Fleischmann
image captioning
visual impairments

Despite significant developments in the world of automated image captioning, current image captioning approaches are not well-aligned with the needs of people with visual impairments. People who are blind or with low vision share a unique and real challenge –their visual impairment exposes them to a time-consuming, and sometimes, impossible task of learning what content is present in an image without visual assistance. As such, these communities often seek a visual assistant to describe photos they take themselves or find online. 

In an ideal world, a fully-automated computer vision (CV) approach would provide such descriptions. However, this artificial intelligence (AI) process is riddled with challenges. Not only is CV work missing images taken by this population, but people who are blind and with low vision are required to passively listen to one-size-fits-all descriptions of images to locate information of interest. In addition, CV algorithms often deliver incomplete or incorrect information. Because of these shortcomings, reliable image captioning systems continue to depend on humans to provide descriptions of photos to people with visual impairments. 

Determined to find a way to improve image captioning for blind and low vision communities, Principal investigator and Texas iSchool Assistant Professor Danna Gurari and Associate Professor Ken Fleischmann believe there is a more efficient and effective solution that reduces human effort and produces accurate results for communities who are blind or with low vision. And they recently embarked on a new project to “design algorithms and systems that close the gap between CV algorithm and human performance for describing pictures taken by both sighted and visually impaired photographers.” 

But the Texas School of Information professors weren’t the only ones thinking about how to improve image captioning for people who are blind or with low vision. A team of researchers at Microsoft Research recently announced a similar vision and goal –to train AI systems to provide more detailed captions that can offer a richer understanding, and more accurate representation of images for the blind or those with low vision. In light of this mission, Microsoft Research developed a new project called the Microsoft Ability Initiative.

According to Microsoft Research Principal Researcher and Research Manager Meredith Ringel Morris, “the companywide initiative aims to create a public dataset that ultimately can be used to advance the state of the art in AI systems for automated image captioning.”

After a competitive process involving a select number of universities, the search for an academic research unit with whom they could partner for the new venture came to an end when Microsoft Research chose The University of Texas at AustinSchool of Information. The proposed work of Gurari and Fleischmann was the only project selected through this competition.

The Texas iSchool research team proposed two main tasks of (1) introducing the first publicly-available image captioning dataset from people with visual impairments paired with a community AI challenge and workshop, and (2) identifying the values and preferences of people with visual impairments –to inform the design of next-generation image captioning systems and datasets. 

“The collaboration builds upon prior Microsoft research that has identified a need for new approaches at the intersection of computer vision and accessibility,” explained Morris.

The companywide initiative aims to create a public dataset that ultimately can be used to advance the state of the art in AI systems for automated image captioning.

The Microsoft Research team which includes Ed Cutrell, Roy Zimmermann, Meredith Ringel Morris, and Neel Joshi, plans to collaborate with UT Austin, School of Information over an 18-month period. Gurari and Fleischmann will lead the UT Austin team, which will also include three PhD students and one postdoctoral fellow.

The Microsoft Ability Initiative builds on the interdisciplinary team’s expertise in computer vision, human-computer interaction, accessibility, ethics, and value-sensitive design. Gurari’s team is experienced in establishing new datasets, designing human-machine partnerships, creating human computer interaction systems, and developing accessible technology. As co-founder of the ECCV VizWiz Grand Challenge in 2018, Gurari is skilled in community-building and has a previous record of success in creating public datasets to advance the state-of-the-art in AI and accessibility.

Fleischmann’s team offers complementary experience in the ethics of AI and understanding users’ values to inform technology design. Given his expertise in the role of human values in the design and use of information technologies, Fleischmann will lead the effort focused on uncovering the needs and values of people with visual impairments –which will ultimately inform the design of future image captioning systems.

The Microsoft researchers involved in this initiative have specialized experience in accessible technologies, human-centric AI systems, and computer vision. “Our efforts are complemented by colleagues in other divisions of the company, including the AI for Accessibility program, which helps fund the initiative, and Microsoft 365 accessibility,” explained Morris.

Dubbed “a collaborative quest to innovate in image captioning for people who are blind or with low vision,” Morris explained that “the Microsoft Ability Initiativeis one of an increasing number of initiatives at Microsoft in which researchers and product developers are coming together in a new, cross-company push to spur innovative and exciting new research and development in the area of accessible technologies.” 

Gurari believes that the initiative “will not only advance the state of the art of vision-to-language technology, but it will also continue the progress Microsoft has made with such tools and resources as the Seeing AI mobile phone application and the Microsoft Common Objects in Context (MS COCO) dataset. It will also serve as a great teaching opportunity for Texas iSchool students.”

The Texas iSchool team will employ a user-centered approach to the problem, including working with communities who are blind or with low vision to improve understanding of their expectations of image captioning tools. The team will also host community challenges and workshops to accelerate progress on algorithm development and facilitate the development of more accessible methods to assist people who are blind or with low vision. 

Gurari and Fleischmann explain that “this work can empower people with visual impairments to more rapidly and accurately learn about the diversity of visual information, while contributing to solving related problems including image search, visual question answering, and robotics.”

The Microsoft Research team launched the new collaboration with the Texas iSchool during a two-day visit to Austin in January. Morris noted that the Microsoft Research team came away from the meeting at The University of Texas at Austin, School of Information, “even more energized about the potential for this initiative to have real impact in the lives of millions of people around the world.” “We couldn’t be more excited,” she said.

The Texas iSchool professors share the Microsoft Research team’s excitement about their upcoming collaboration. “To be selected for this gift is a great honor,” said Gurari and Fleischmann. “We look forward to working with the Microsoft Research team over the months, and are eager to make progress with our shared goal –to better align image captioning systems with the needs of those who are blind or with low vision.” 

glqxz9283 sfy39587stf02 mnesdcuix8