I have the same problem when I'm watching lillyhammer. In dark areas it flickers like it's having a loose cable or something. Some dark scenes in Arrow as well. I think it's a netflix problem, not internet.
The streaming quality on the Xbox One is crap. Been watching Orange is the New Black and the Walking Dead. It's terrible. I switch to the PS3/4 and it's fine. I've seen many other threads stating the same. I'm sure they'll patch it eventually, but something is up currently.