In 1764, the order of Duke Francesco III d'Este to reunite the Works Pie of the city of Modena, involved the construction in Piazza Sant'Agostino of a "Grande Albergo dei Poveri" in which to welcome them. This operation is understood in a more general policy of social reform and renewal to cope with the numerous problems of public order and rehabilitation of the city, as already happened in 1753 with the construction of the Great Hospital facing the same square. The first nucleus of the "Albergo dei Poveri", designed by the Modenese architect Pietro Termanini, was obtained in the liberated spaces of the Arsenale Estense, followed by those of the Convent of the Augustinians and the adjoining church. The aims of the new construction, which ended in 1771, were both assistance and education, providing for the sustenance of the dispossessed and their employment. In 1788 the Duke Ercole III d'Este, successor to Francesco III, transformed the "Grande Albergo" into "Albergo delle Arti", welcoming you the teaching of activities close to the disappearance, among which those of the processing of fabrics. Over the years the building experienced seasons of changes and transformations: from the suspension of subsidies to the indigent who worked there following the arrival of the Napoleonic troops, to the place of military residence of Este officers, then to the seat of the Provincial Hospitality of Mendicità.
After the Unification of Italy, the desire to bring together the various cultural and conservative institutes of the city within the Hotel Arti, led to the signing, in 1868 in Florence, of the first stipulation between the Italian Government and Archduke Francis V d'Este to put the Este Museum in it, so as to sell the Palazzo Ducale to the Military School. In 1881 the building was then purchased by the City of Modena, which placed the Civic Library of Art Poletti, the Municipal Archive and the Civic Museum.