Use Clustering to Improve Resource Forecasting
Humans partition everything they interact with into groups—they spend their waking hours clustering information about the world into little siloes, organizing things in ways that help them get by. These groupings, although often imperfect and subject to all manner of hidden psychological biases, help people to profile the world around them and make the best possible judgments about how to go about their lives.
In construction, a similar kind of data-driven and experiential profiling can be undertaken with the aid of a suite of clustering algorithms. These algorithms are helpful tools for boosting system and process performance, whether it be day-to-day resource planning or providing portfolio planners the high-fidelity capacity data they need to make business critical decisions.
What is clustering?
Clustering is an unsupervised machine learning technique that, in general, aims to partition data in such a way that like observations are grouped together. The more similar two observations are, the more probable they are to be placed in the same group. Likeness, in a clustering sense, is determined by measures of geometric closeness.
Now apply that to resource forecasting.
Forecasting project resources such as labor, material or equipment is heavily dependent on the detail and accuracy of the project schedule and project estimate. This creates a challenge when trying to balance demand and supply across an organization. Clustering provides a data-driven framework that can be applied to the process to enhance accuracy and improve efficiency.
Data is the currency of effective machine learning analysis, and more currency is always preferred to less. Projects, trades, vendors, project managers, finance and contracts have information locked away in various enterprise resource planning systems—information including metrics (lagging and leading indicators) and other attributes that can help boost the performance of clustering analysis. Often, this information is incredibly rich. It always pays to cast a wide net when collecting data for clustering, as one never quite knows where key variables might be hiding.
Many areas of a major capital project are good candidates for clustering. Some of the most relevant include the following:
- Project Management: Develop an in-house profile of staff capabilities or performance.
- Vendors: Determine which types of vendors are contracted for which types of work and how well they perform.
- Estimates: Examine how closely a proposed capital project aligns with the known cost and returns of similarly completed capital projects.
- Asset Management: Transform maintenance logs and other digital exhaust from operations to identify work process trends and optimize maintenance scheduling.
ESTABLISHING BENCHMARKS FOR FUTURE PROJECTS
Clustering is great for exploring data, developing profiles and validating assumptions. With this in mind, project metrics is a great place to start.
Today, a project manager might cluster project performance metrics into three ordered groups (red, yellow or green). This framework would then be leveraged to organize and apply different management activities to a project (e.g., holding a special meeting on a red project to promote awareness of the challenges being faced).
The precise boundaries of a group—the criteria that makes one project red and another yellow—are often based on the individual expertise and experience of a given project manager and can vary from person to person. It can be tough to benchmark against this type of heuristic decision making. Applying clustering algorithms can help an organization dynamically quantify and evaluate the precise thresholds needed to optimize the application of management practices.
The groupings in a clustering analysis can continually be refined as new data or new variables come to light. Importantly, a clustering analysis can show that, in fact, the data are best described by having more (or fewer) than three clusters. Clustering thresholds are not set in stone; they evolve with the organization. This can help an organization move from the standard stoplight to something a little more colorful (and better aligned with management best practices along the way).
CLUSTERING FOR THE FIRST TIME
For those going down the clustering path for the first time, be prepared for some unavoidable elbow grease. Data cleaning and preparation are often the most time-consuming phases and can easily take 80 percent of the total time allocated for the exercise. It is usually necessary to wrangle with the data for quite a while before it’s in just the right format for analysis.
The most important challenge in any type of clustering analysis is determining the right number of clusters. Too many and the groupings become granular and unhelpful, too few and valuable information is masked. Choosing the number of clusters cannot be done in a vacuum; it requires precise domain knowledge and experience in addition to the required technical know-how. In this vein, clustering works best when the data science and capital projects teams work in concert.
USING DATA TO MAKE INFORMED COMPARISONS
A clustering profile can help a decision maker identify those projects that are most similar and should be compared to each other for performance analysis and reporting. This ensures that valid apples-to-apples comparisons are being made.
Evaluating clustering analyses over time is mainly a task of monitoring changes in the shape and number of clusters. If users are seeing clustering profiles that shift significantly over time, the data is probably not stable or settled. It’s likely that things are in flux and that no reliable trends have emerged.
On the other hand, if the clusters remain relatively constant, it is a good bet that there are key relationships in the data that are likely to be pretty robust. Stability is always preferred, but identifying changes early before they become entrenched can be a great way to proactively manage a portfolio.
There is untapped potential in data—potential that can be used to enhance and improve work. Without a data-informed approach to forecasting, an organization can easily overlook the historical productivity of its resources. Employing clustering analyses can provide a validity check on long-held assumptions and act as an input variable for future planning.
Written by Michael Goggin, Enstoa
An experienced software engineer and project controls expert, Michael Goggin applies systems engineering principles, cost control and project management principles to design and configure large, complex software implementations. At Enstoa, he works closely with clients worldwide to design smart technology solutions that support their organizational growth. For more information, visit www.enstoa.com.
Written by Michael Matošin, Enstoa
Michael Matošin is a machine learning, data science, and operations research specialist at Enstoa. Michael leverages the most up-to-data models and methods to inform key business decisions and drive organizational value. A significant focus of his work is developing predictive models and analytics in order optimize systems and processes to deliver dynamic, best-in-class solutions. For more information, visit www.enstoa.com.
Link to the original article here.
Published in Construction Executive on August 23, 2018
Humans partition everything they interact with into groups—they spend their waking hours clustering information about the world into little siloes, organizing things in ways that help them get by. These groupings, although often imperfect and subject to all manner of hidden psychological biases, help people to profile the world around them and make the best possible judgments about how to go about their lives.
In construction, a similar kind of data-driven and experiential profiling can be undertaken with the aid of a suite of clustering algorithms. These algorithms are helpful tools for boosting system and process performance, whether it be day-to-day resource planning or providing portfolio planners the high-fidelity capacity data they need to make business critical decisions.
What is clustering?
Clustering is an unsupervised machine learning technique that, in general, aims to partition data in such a way that like observations are grouped together. The more similar two observations are, the more probable they are to be placed in the same group. Likeness, in a clustering sense, is determined by measures of geometric closeness.
Now apply that to resource forecasting.
Forecasting project resources such as labor, material or equipment is heavily dependent on the detail and accuracy of the project schedule and project estimate. This creates a challenge when trying to balance demand and supply across an organization. Clustering provides a data-driven framework that can be applied to the process to enhance accuracy and improve efficiency.
Data is the currency of effective machine learning analysis, and more currency is always preferred to less. Projects, trades, vendors, project managers, finance and contracts have information locked away in various enterprise resource planning systems—information including metrics (lagging and leading indicators) and other attributes that can help boost the performance of clustering analysis. Often, this information is incredibly rich. It always pays to cast a wide net when collecting data for clustering, as one never quite knows where key variables might be hiding.
Many areas of a major capital project are good candidates for clustering. Some of the most relevant include the following:
- Project Management: Develop an in-house profile of staff capabilities or performance.
- Vendors: Determine which types of vendors are contracted for which types of work and how well they perform.
- Estimates: Examine how closely a proposed capital project aligns with the known cost and returns of similarly completed capital projects.
- Asset Management: Transform maintenance logs and other digital exhaust from operations to identify work process trends and optimize maintenance scheduling.
ESTABLISHING BENCHMARKS FOR FUTURE PROJECTS
Clustering is great for exploring data, developing profiles and validating assumptions. With this in mind, project metrics is a great place to start.
Today, a project manager might cluster project performance metrics into three ordered groups (red, yellow or green). This framework would then be leveraged to organize and apply different management activities to a project (e.g., holding a special meeting on a red project to promote awareness of the challenges being faced).
The precise boundaries of a group—the criteria that makes one project red and another yellow—are often based on the individual expertise and experience of a given project manager and can vary from person to person. It can be tough to benchmark against this type of heuristic decision making. Applying clustering algorithms can help an organization dynamically quantify and evaluate the precise thresholds needed to optimize the application of management practices.
The groupings in a clustering analysis can continually be refined as new data or new variables come to light. Importantly, a clustering analysis can show that, in fact, the data are best described by having more (or fewer) than three clusters. Clustering thresholds are not set in stone; they evolve with the organization. This can help an organization move from the standard stoplight to something a little more colorful (and better aligned with management best practices along the way).
CLUSTERING FOR THE FIRST TIME
For those going down the clustering path for the first time, be prepared for some unavoidable elbow grease. Data cleaning and preparation are often the most time-consuming phases and can easily take 80 percent of the total time allocated for the exercise. It is usually necessary to wrangle with the data for quite a while before it’s in just the right format for analysis.
The most important challenge in any type of clustering analysis is determining the right number of clusters. Too many and the groupings become granular and unhelpful, too few and valuable information is masked. Choosing the number of clusters cannot be done in a vacuum; it requires precise domain knowledge and experience in addition to the required technical know-how. In this vein, clustering works best when the data science and capital projects teams work in concert.
USING DATA TO MAKE INFORMED COMPARISONS
A clustering profile can help a decision maker identify those projects that are most similar and should be compared to each other for performance analysis and reporting. This ensures that valid apples-to-apples comparisons are being made.
Evaluating clustering analyses over time is mainly a task of monitoring changes in the shape and number of clusters. If users are seeing clustering profiles that shift significantly over time, the data is probably not stable or settled. It’s likely that things are in flux and that no reliable trends have emerged.
On the other hand, if the clusters remain relatively constant, it is a good bet that there are key relationships in the data that are likely to be pretty robust. Stability is always preferred, but identifying changes early before they become entrenched can be a great way to proactively manage a portfolio.
There is untapped potential in data—potential that can be used to enhance and improve work. Without a data-informed approach to forecasting, an organization can easily overlook the historical productivity of its resources. Employing clustering analyses can provide a validity check on long-held assumptions and act as an input variable for future planning.
Written by Michael Goggin, Enstoa
An experienced software engineer and project controls expert, Michael Goggin applies systems engineering principles, cost control and project management principles to design and configure large, complex software implementations. At Enstoa, he works closely with clients worldwide to design smart technology solutions that support their organizational growth. For more information, visit www.enstoa.com.
Written by Michael Matošin, Enstoa
Michael Matošin is a machine learning, data science, and operations research specialist at Enstoa. Michael leverages the most up-to-data models and methods to inform key business decisions and drive organizational value. A significant focus of his work is developing predictive models and analytics in order optimize systems and processes to deliver dynamic, best-in-class solutions. For more information, visit www.enstoa.com.
Link to the original article here.
Published in Construction Executive on August 23, 2018
