Do you know how your NAT settings are? These can be seen under your Network Settings on the Xbox One, you want them to be Open. They have to be modified on your computer though. If it is Strict then you should try to open the ports for Xbox Live.
Are you using your X1 without the Kinect plugged in? I notice that as soon as I attempt to resume my game, I'm automatically logged out of my Profile and in turn, the EA servers. Only way to get around this is to Quit the BF4 application and go back to into it.