Bailey's early death must have meant tough times for his widow and her many children one of whom, William, became a pastor in Datchet, Oxfordshire. He wrote in a letter of the religious education his mother gave him, "owing to which," he observes "I was kept, by the grace of God, from many snares and temptations to which others have been a sacrifice." He served an apprenticeship with a grocer and draper at Bedworth, Warwickshire, before removing to a situation at Gosport, Hampshire, where he came under the ministry of David Bogue (1750-1825). It was through Bogue that he was saved, although hardly even aware of it at the time.
From Gosport he moved to Henley and then to London, where he was baptised by William Smith (1749-1821) of Eagle Street, on October 9, 1796, then aged 25.
He did not join the church at this time and there is a gap in his history until 1811 when "moved by a weakly state of health and a growing sense of the importance of eternal things, he began to record many of the exercises of his soul, and keep a strict watch over his heart." He records seven years of domestic happiness and a prosperous run of business, thankfully but expresses much concern lest these should lead him astray.
At this time he belonged to the Independent church in Windsor but was increasingly unhappy about not being in a Baptist church. He eventually resigned while still maintaining a friendship with the church and its pastor, Alexander Redford (1759-1840). He then joined the Baptist church in Datchet under John Young from Staines, soon becoming a deacon. He began to preach from time to time and in 1815, when Young stepped down because of illness, he became the regular preacher, being ordained in 1819. Although not blessed with great success he was enabled to sustain the work.
In 1832 he wrote the Association letter. Ill health only interrupted his ministry seriously late in 1843,. The church was able to install his successor (John Tester) before Bailey's death, which came on June 30, 1844.