In the 1970s, many Malay Muslims living in areas such as Buangkok, Jalan Kayu and Tongkang Pecah (present-day Fernvale) were resettled in parts of Ang Mo Kio. The nearest places of worship for these residents then were in Upper Serangoon, Sembawang Road and Thomson Road. Within Ang Mo Kio, religious sermons would be delivered at Block 715 Avenue 6 and prayers during festivities such as Hari Raya were conducted at a former community club at Block 208 along Avenue 1.
The idea of building a mosque in Ang Mo Kio was mooted during the late 1970s in the house of a Malay Muslim, Haji Muhammad Ariff, at Nemesu Avenue. Following that, a building committee was formed and Muslims living in and around Ang Mo Kio began fund-raising efforts. A 3,000 m2 site was secured along Ang Mo Kio Avenue 6 and the ground-breaking took place in 1979. In 1980, the mosque was finally completed at a cost of close to $1.8 million.
Opened on 21 September 1980, the mosque has served as the centre of Islamic activities for Muslim residents in Ang Mo Kio and its surroundings ever since. On top of its religious function, Masjid Al-Muttaqin also runs a kindergarten, marriage preparatory classes and counselling sessions.