In 1856 a number of the inhabitants were moved to Norfolk Island because of the risk of famine but returned to Pitcairn between 1858 and 1864.

In 1898 the island was placed under the jurisdiction of the High Commissioner for the Western Pacific. Responsibility for administration was transferred to Fiji in 1952, but the island remained a Crown Colony. At its peak, the population reached about 250 but was down to 100 in the 1970s.

Postal History
With such a small and largely illiterate population, early letters are very uncommon. In the 1880s letters were sent by visiting ships, often American whalers and the missionary steamer Pitcairn. These are known with a straight line 'Pitcairn Island' usually struck in the lower left-hand corner. However, this was discontinued in 1900 when the service by the Pitcairn was withdrawn.

In 1921-6, Britain and New Zealand agreed to accept mail from the island. Handstamps were used inscribed 'Posted at Pitcairn Island - No stamps available'. In 1927 a New Zealand PA was opened and current New Zealand stamps were used until the stamps of the island were issued in 1940.