Ashby provided the Astros with ten seasons of fine catching, quality game calling, and respectable hitting. Houston often tried to displace him with prospects like George Bjorkman, Luis Pujols, John Mizerock, and Mark Bailey, but Ashby beat off successive challenges to keep the regular job year after year. It finally took a bad back and Craig Biggio to send Ashby packing; he was released in 1989 after refusing a trade to Pittsburgh.
From 1979 into 1985, Ashby was battered by Joe Niekro's knuckleballs. "All the broken fingers and broken toes I can remember were caused by the knuckler," Ashby said. He was disabled five times with Houston.