The region of Silves has been inhabited since the Palaeolithic, as attested by archaeological remains, including several menhirs. The river Arade, which was navigable in historical times, linked the hinterland to the open ocean and allowed the transport of produce and commerce. The town of Silves (Cilpes) was possibly founded during the times of Roman domination, when the region was part of the Lusitania province. It was probably a Lusitanian Castro in pre-Roman times; however the region was also settled by other Indo-European tribes, such as the Celtici and Cynetes (or Conii). Silves was also part of the Visigothic Kingdom.

After 713, when the Moors invaded Iberia, Silves became part of the Umayyad Emirate of Córdoba under the Arabic name of Shilb. In the 10th century it was one of the most important towns in western Al-Andalus. Silves became an independent taifa in 1027 under the rule of Ibn Mozaine and his son, who was dethroned in 1051 by al-Mu'tadid, the governor of Seville. al-Mu'tamid ibn 'Abbad, the son of al-Mu'tadid and a famous poet, ruled the taifa of Silves until 1091. After the Almoravid conquest the town became Almohad in 1156.

In 1189, King Sancho I of Portugal conquered (in the Reconquista) the town with the aid of Northern European crusaders. Sancho ordered the fortification of the city and built a castle, which is today an important monument of Portuguese heritage. At the time he also styled himself "By the Grace of God, King of Portugal and Silves (Dei Gratiæ, Rex Portugalliæ et Silbis). However, he soon lost the town again to the Almohads. Periodic raiding expeditions were sent from Al-Andalus to ravage the Iberian Christian kingdoms, bringing back booty and slaves. The governor of Córdoba attacked Silves in 1191, and took 3,000 Christian slaves. Again under Muslim rule, the city would then prosper to the point of being called the Baghdad of the West.

The town was finally taken from the last Muslim king Ibn Afan by Paio Peres Correia, Grand-Master of the Order of Santiago in 1242, after the Alentejo and most of the coast had already fallen in 1238. The great mosque was changed into Silves Cathedral (Sé Catedral). Silves declined in importance thereafter and was eclipsed in the region by Faro during the colonial period. In 1491, the town of Silves was given to queen Eleanora by her husband, king John II of Portugal.