That is one of the findings in a national survey run by the Pew Research Center and Smithsonian Magazine. From the Pew Research article that summarizes the survey:
The legal and regulatory framework for operating non-military drones is currently the subject of much debate, but the public is largely unenthusiastic: 63% of Americans think it would be a change for the worse if “personal and commercial drones are given permission to fly through most U.S. airspace,” while 22% think it would be a change for the better. Men and younger adults are a bit more excited about this prospect than are women and older adults. Some 27% of men (vs. 18% of women), and 30% of 18–29 year olds (vs. 16% of those 65 and older) think this would be a change for the better. But even among these groups, substantial majorities (60% of men and 61% of 18-29 year olds) think it would be a bad thing if commercial and personal drones become much more prevalent in future years.
The poll itself is pretty interesting and addresses public opinion on a wide variety of prospective technological developments, but I think that the drone findings are of particular importance, given the wide array of legislative developments regarding drones in recent years.
The poll did not seem to compare people's opinions of domestic use of government drones to the use of personal and commercial drones, but I wish that it had. My instincts tell me that approval ratings of government drones would be even lower than the ratings for personal and commercial drones. I base this assertion on my observation that lately, more state restrictions on drones have solely targeted government drones, or have placed far more detailed and heavy restrictions on government drones, compared to privately operated drones.
While the poll may not have explored this distinction, I am fairly certain that an answer will become more apparent as more states continue to enact drone regulations.