Yes, there is a time when he remembers that Robert never forgets anything, so if she stayed in the Kingdom no amount of time would save her. Robert is still as mad about Lyannas death as he was when it hapened. Robert and Stannis are alike in this. SO that is why he knew children would not be safe, Ned is too honorable to like or respect Cersei.