It was soon realised that Botany Bay did not live up to the glowing account that the explorer Captain James Cook had provided. The bay was open and unprotected, fresh water was scarce, and the soil was poor. First contacts were made with the local indigenous people, the Eora, who seemed curious but suspicious of the newcomers. The area was studded with enormously strong trees. When the convicts tried to cut them down, their tools broke and the tree trunks had to be blasted out of the ground with gunpowder. The primitive huts built for the officers and officials quickly collapsed in rainstorms. The marines had a habit of getting drunk and not guarding the convicts properly, whilst their commander, Major Robert Ross, drove Phillip to despair with his arrogant and lazy attitude. Crucially, Phillip worried that his fledgling colony was exposed to attack from the Aborigines or foreign powers.
On 21 January, Phillip departed the Bay in three small boats to explore other bays to the north. Phillip discovered that Port Jackson, about 12 kilometres to the north, was an excellent site for a colony with sheltered anchorages, fresh water and fertile soil. Cook had seen and named the harbour, but had not entered. Phillip's impressions of the harbour were recorded in a letter he sent to England later; "the finest harbour in the world, in which a thousand sail of the line may ride in the most perfect security ...". The party returned on 23 January.
On 26 January 1788, the fleet weighed anchor and sailed to Port Jackson. The site selected for the anchorage had deep water close to the shore, was sheltered, and had a small stream flowing into it. Phillip named it Sydney Cove, after Lord Sydney the British Home Secretary. This date is still celebrated as Australia Day, marking the beginnings of the first British settlement. The British flag was planted and formal possession taken. This was done by Phillip and some officers and marines from Supply, with the remainder of Supply's crew and the convicts observing from on board ship. The remaining ships of the Fleet did not arrive at Sydney Cove until later that day.