They say dad Kevin Krim was away on a trip and their mother found them in a bathtub after returning home with another child.
Police say mom Marina Krim arrived at the dark apartment near Central Park on Thursday evening and initially thought the nanny had taken the children out. He says when she looked in the bathroom she saw 1-year-old Leo and 6-year-old LuLu dead.
The nanny was found with stab wounds to her neck, and a kitchen knife was nearby, police said. There was no water in the bathtub, they said.
"Both children were taken to St. Luke's Hospital where they were pronounced dead," said NYPD Deputy Commissioner Paul Browne said, as reported by WCBS 880.
Music therapist Rima Starr, who lives on the same floor as the family, said she heard screams coming from their apartment at around 5:30 p.m.
"There was some kind of screaming about, `You slit her throat!' It was horrible," she said.
She said she believed the nanny had been hired just recently.
"I met her in the elevator, the day before yesterday, and was making small talk," she said.
After police arrived, she said, the mother remained in the building's lobby, screaming hysterically and clutching her surviving child.
The nanny was in police custody and was hospitalized in critical condition.
The apartment building sits in one of the city's most idyllic neighborhoods, a block from Central Park, near the Museum of Natural History and blocks from Lincoln Center for the Performing Arts. A drugstore is in the bottom of the building.