Both instruments accomplish the same task of determining latitude. Latitude, for the uninitiated, is the distance you are North or South of the Equator. Latitude may be found by measuring the angle between the horizon and the sun at its highest point (zenith). There are many ways to do this, and it may be as simple as the use of a quadrant or cross-staff. Neither of those instruments are as accurate as the backstaff, which was sometimes called the Davis Quadrant for its sixteenth century inventor John Davis.
The photograph above was generously provided by E.T. Fox, curator of the Golden Hind Museum in London. The backstaff he holds on the quarterdeck is one that he constructed himself, and you can see more examples of his work here. In this photograph you can see how the staff is used. In order to measure the elevation of the sun from the horizon, you would have to turn your back to the sun, as demonstrated in this video by the Mariners' Museum and Park in Newport News, Virginia.
The biggest drawback of a backstaff is that it is nearly impossible to use for sighting anything but the sun. The backstaff is practically useless for measuring the altitude of celestial bodies at night. This lack contributed to its eventual replacement in the 1730's by the octant. A more accurate instrument, the octant could also measure any celestial body visible in the sky, night or day.
In the mid to late eighteenth century, these instruments co-existed in a period of transition that saw the phasing out of the backstaff. This phasing out is evidenced by cartoons of the period, such as "Ridiculous Taste or The Ladies Absurdity," a 17771 print by Matthew Darly. The version below is in the collection of the British Museum.
Lampooning the tall wigs piled onto the heads of fashionable ladies, Darly is probably not acquainted with celestial navigation as more than just a concept. As part of the joke, he has one dandy gentleman taking a sighting by the wig of the seated woman.
Our macaroni holds a sort of octant (lacking many of the necessary pieces), but he holds it as one would operate a backstaff. Darly is either unaware of precisely how an octant works and so falls back on the general appearance of a navigator working a backstaff, or is intentionally making the scene more ridiculous through the use of an oafish fop who doesn't quite understand what he's doing.
Either way, it is a piece of art that exists in the time of transition between the dominant technologies in determining latitude.
The same is true of R. Attwold's cartoon "The Naval Nurse, or Modern Commander." The first image here is a 1740-1750 sketch from the British Museum's collection, used as a study for the final piece which was printed in 1750. The final engraving featured here comes from the Yale University Lewis Walpole Library.
On the wall of the Boy Captain's cabin is a series of pistols, a blunderbuss, a telescope, and a backstaff.
In the final print, however, the instrument is changed from a backstaff to an octant.
Attwold's point in this cartoon is the youth and inexperience of naval officers. Such young men would be much more open to using newer technologies and newer methods of navigation. Perhaps it is this which accounts for Attwold's choice to replace the original backstaff with the octant.
Just as Attwold erased the backstaff from his image in favor of the octant, the octant would eventually erase the backstaff from the list of essential navigational instruments. The octant itself shortly evolved into the sextant, which remains the primary means of celestial navigation to this day. While the GPS has largely supplanted the use of celestial navigation, sextants can still be found in marine supply stores around the globe.