Yeast is used symbolically in the Bible to represent influence, often negatively. Just as a little bit of yeast, permeates dough and produces a noticeable result, so does influence, be it good or bad.
In the Books of Moses, there are thirty-two references to not using yeast (and more for it’s counterpart, leaven) in various religious practices, which symbolically shows the removal of sin. The theme continues in Judges 6:19, I Samuel 28:24, and Ezekiel 45:21. (Leviticus 7:13 and 23:17 are two exceptions, when yeast should be used.)
Paul compares false teaching to yeast (Galatians 5:9).
Another negative connotation is when Jesus says to beware of the yeast of the Pharisees, which is hypocrisy (Luke 12:1), as well as the yeast of Sadducees (Matthew 16:6-12) and the yeast of Herod (Mark 8:14-15).
Another reference to yeast is in 1 Corinthians 5:6-8, where Paul uses yeast as a metaphor for boasting. Paul says to get rid of bread with old yeast (malice and wickedness) and use new bread without yeast (sincerity and truth).
In likely the only positive usage of yeast in the Bible, Jesus says that the Kingdom of God (Heaven) is like yeast (Matthew 13:33 and Luke 13:18-21).