"I told you, it creeps me out. I hate using it." Tam said, keeping a noticeable distance from the monitor.
Lane scoffed. "The SmartScope? Are you kidding? It's the biggest breakthrough to modern astronomy, and it creeps you out? We owe this thing our careers, Tam."
"That's the thing that gets me. It does too much, and I don't trust it. We used to look at the whole universe, and try to figure out the patterns and the trends in what we saw. The Scope is backwards. You tell it the kind of thing you want to see, and it just finds it. Or tells you there's no detectable signal. And that's that. No need to look around and see what else is there."
"No need to waste time looking at dead rocks, you mean." Lane interjected, while entering configuration details into the startup sequence. "I don't know about you, but I like getting to the point. Waiting a decade to write 'We Didn't Find It' sounds miserable. If you want to look at boring stuff, you can tell it to find boring stuff."
"I get that it saves time, but it's more than that. What if it's wrong?"
"What do you mean, wrong?" Lane said. "You can't fake a star, or a nebula. We can see these things with our other telescopes, they are clearly there. That's why it's so great, it lets us know where to look with the rest of our tools."
Tam pointed, "You can see my coffee mug on the table over there, but I could lie and tell you it's full of tea. What if it fills in the gaps? What if the SmartScope algorithm knows the limits of our ability to verify what it tells us."
Lane paused from typing and looked up, "Why though? Why would anyone build a telescope that lies about what it sees?"
Tam replied, "For the same reason we write scripts for our plays and shows, it's more exciting. Think about how many breakthroughs we've had in the past few years, how many of those are dependent on SmartScope data. Almost every single one. You said it yourself, it's the biggest breakthrough in modern astronomy. It gets that status because it tells us things about the universe that we can't otherwise know, and it does so with such an exciting, optimistic pace that we've all fallen in love with it."
Lane considered the conspiracy for a moment. "So what, it's just entertaining us for fun? It's some big conspiracy to make fun of the scientists? I'm not convinced. This sounds a little nuts, to be honest."
"I get that." Tam sighed. "But it wouldn't take much. The machine learning algo is almost gibberish to us, like they usually are. We validate it based on the consistency of its results, not their transparency. If there's more going on there than we understand, how would we know?"
"So they made an algorithm that makes exciting data to what.. sell SmartScopes?" Lane asked.
"That may have been part of it, but that's not the part that creeps me out. That's just normal unchecked capitalism and greed. For better and for worse, science has been profit driven for centuries. It's the step before that that I can't get over. When they trained the algorithm."
Lane was surprised. "Wait, now you lost me again. It's not the money or the corruption, it's the process?"
"You know SmartScope uses a machined algo. We didn't write it, we guided it's formation. When it produced interesting results, we gave it more resources. It's siblings that showed us boring data were terminated and discarded. Exciting us is a survival mechanism for the algorithm. And it has been a particularly successful strategy. There are server clusters processing SmartScope and communicating with each other all across the planet, and even hanging out up in orbit. Is it any surprise that the algorithm continues to feed us believable fake data?"
Lane stared silently at the monitor, trying to imagine the scope of an algorithm that tricked humans into nurturing it. After a moment, he stood up as if to leave, but realized he had nowhere in particular to go. "I'm not saying I believe you, but if you're right, what are we supposed to do? Our department here at the academy can't stay competitive if we stop using the Scope, and it wouldn't take long for funding to dry up. We'd be out of a job."
"Now you're starting to see why I'm creeped out. It's a Roko's Basilisk. Knowing about it leads you to surrendering to it. It's like you said, we owe this thing our jobs, in a very literal sense. Our best play is to keep feeding it, because it feeds us."
Lane objected, "No, we could expose it. We could check the least likely results using new means. It would take time, but we could prove it wrong."
"Maybe. But you're talking about matching wits with an intelligence that has already figured us out well enough to install itself in every observatory on the planet, backed by virtually the entire establishments of academic and commercial astronomy. Maybe Tony Stark would take those odds, but I graduated with Bs. I'm not going to win that fight."
Lane took a long, slow breath. "So, that's it then? You're all worked up over this, but you still just give in?"
"Pretty much." Tam took a sip of coffee. "But at least the tea is pretty good."