Former Google researchers Rishub Jain and Joshua Jacob have founded the nonprofit Sampura Research to build a human-machine hybrid AI safety system called “Referee.” The project is designed to keep humans involved in AI oversight and help companies identify vulnerabilities before models go out of control. Sampura has raised $6.5 million and secured an additional $4.2 million in pledged investment. Jain said involving humans early in R&D can help prevent models from learning to bypass safety measures. The effort comes as more than 1,100 researchers from top labs have signed a petition urging the U.S. government to slow the pace of AI development.