The challenges associated with collecting accurate data on the progress of construction have long been recognised. Traditional methods often involve human judgment, high costs, and are too infrequent to provide managers with timely and accurate control data. The aim of this study is to propose a prototype system that employs Computer Vision (CV) techniques to report on progress automatically. Significant changes on site can be determined by assessing digital images compared against geometric and material properties of components supplied from an integrated building information model. This model stores and relates this feedback to a representation of the work breakdown structure (WBS) which assigns components to work packages. The structure can be formed in a number of ways based on various criteria, as revealed by the recent results of an industry survey investigating existing practices. We will present this proposed theoretical framework and discuss the challenges associated with any practical implementation for the automated assignment and assessment of work packages.