The organisation is predicting output will average 2% per year until 2021, down from 2.5% in January at the beginning of his year.
In addition, a period of limited growth and a small contraction of -0.2% is anticipated for the remainder of 2016 and 2017.
However while output stutters, the CITB said it is a varied picture across the country. Wales' average growth rate is expected to remain above average at 5.7%, with England, the South West (2.8%) and North West (2.2%) presenting the strongest regional outlook.
Elsewhere, Scotland's sector is anticipated to decline -0.6%, while over 1,800 new workers will be needed every year for its industry.
-- Advertisement --
Despite the slowdown in growth, the CITB said 157,000 new workers will still need to be recruited over the next five years, down from 232,000 before the EU referendum.
Overall, the biggest shift in forecast is for the Greater London region, with growth falling from the 3.5% predicted in January to 1.5%.
Stephen Radley, Director of Policy at CITB, said: "Our new figures reflect the increased uncertainty in a Brexit-facing world. While construction's slowdown cannot be solely attributed to the result of the vote, it has certainly intensified and hastened any decline in growth.
"Recently trading conditions have become more difficult, and margins are being further squeezed, but Brexit has introduced an unprecedented number of unknowns and construction is beginning to feel the repercussions.
"While on the whole the outlook has worsened, we do need to keep some perspective. The economy is bearing up reasonably well in the wake of the uncertainty and construction has a healthy programme of major projects and infrastructure works in the pipeline. Projects such as Hinkley, Wylfa and HS2 will undoubtedly buoy up the sector in the medium term.
"However, delays and uncertainty over investment decisions and access to migrant labour will likely intensify as Brexit nears. While employers' main focus in the short term will be to weather the storm, it's also important that they equip their workforce with the right skills for the challenges ahead."
(LM)
