Cloud cameras (all sky imagers/ASIs) can be used for short-term (next 20 min) forecasts of solar irradiance. For this reason, several experimental and operational solutions emerged in the last decade with different approaches in terms of instrument types and forecast algorithms. Moreover, few commercial and semi-prototype systems are already available or being investigated. So far, the uncertainty of the predictions cannot be fully compared, as previously published tests were carried out during different periods and at different locations. In this study, the results from a benchmark exercise are presented in order to qualify the current ASI-based short-term forecasting solutions and examine their accuracy. This first comparative measurement campaign carried out as part of the IEA PVPS Task 16 (https://iea-pvps.org/research-tasks/solar-resource-for-high-penetration-and-large-scale-applications/). A 3-month observation campaign (from August to December 2019) took place at Plataforma Solar de Almeria of the Spanish research center CIEMAT including five different ASI systems and a network of high-quality measurements of solar irradiance and other atmospheric parameters. Forecasted time-series of global horizontal irradiance are compared with ground-based measurements and two persistence models to identify strengths and weaknesses of each approach and define best practices of ASI-based forecasts. The statistical analysis is divided into seven cloud classes to interpret the different cloud type effect on ASIs forecast accuracy. For every cloud cluster, at least three ASIs outperform persistence models, in terms of forecast error, highlighting their performance capabilities. The feasibility of ASIs on ramp event detection is also investigated, applying different approaches of ramp event prediction. The revealed findings are promising in terms of overall performance of ASIs as well as their forecasting capabilities in ramp detection.