The son of Colonel H.G. Toler, breeder of trotting horses, Sidney acted on stage by the time he was seven years old. He was an established star of the theatre by the 1890's, long before his career in motion pictures. He was also active as a playwright and had a good enough voice to be cast as a lead baritone with an operatic stock company at the Orpheum Theatre in New York. He made his Broadway debut in 'The Office Boy' in 1903 and for the next nine years went on the road with his own touring acting troupes. His prowess as a writer equalled that of his performances, with two of his plays opening on Broadway, while a third ('The Man They Left Behind') was enacted by no less than eighteen different stock companies in a single week nationwide.
Frequently under the auspices of theatrical impresario David Belasco, Sidney starred in Broadway comedy for twelve years (1918-30). Abandoning his successful stage career, he made the move to Hollywood and played supporting roles as a free-lance actor for several years, often as police officers, bankers or butlers. In the mid-1930's, he joined 20th Century Fox under contract. The death of 'Charlie Chan' impersonator Warner Oland in 1938 presented him with an opportunity for a leading role, and he successfully auditioned for the part among 34 candidates screen-tested.
His expansive, avuncular personality made Sidney, arguably, the most popular incarnation of the famous oriental detective, with a strong line in sarcastic wit, usually directed at 'Number One Son'. He played Chan more than 25 times, beginning with Charlie Chan in Honolulu (1938), and ending with The Trap (1946). However, after 1944, the quality of the series began to decline. With the Second World War causing overseas markets to dwindle, Fox had sold the character rights to 'poverty row' dweller Monogram, which effectively meant poorer scripts and lower production values. Moreoever, after years of being typecast and given few opportunities to expand his range as an actor, Sidney's performances also became less defined and more automatic. During filming of the last three 'Charlie Chan' instalments, Sidney became increasingly incapacitated due to ill-health, resulting in extra screen time for his co-stars Mantan Moreland and Victor Sen Yung. After being bedridden for several months, he passed away at his Hollywood home from intestinal cancer in February 1947.