I think people underestimate how good feedback from non domain experts can be. They’re usually free from the kinds of assumptions everyone in your field already makes, meaning they will just ask the “dumb” questions. And personally, I’ve been surprised by how many supposedly dumb questions I struggle to answer, and they tend to further solidify my foundations by thinking through them.
Non-scientist, so completely unqualified here. But I have a theory: that lots of systems don't work the way you might think they do. Peer review looks like one of those: it looks like a system where other scientists make meaningful comments on your work. But in fact it's just a way of making sure that *someone* has read the paper, which prevents scientists from just writing nonsense.
There is a story in the Economist about how the number of fake papers is rising, and it points out that there's a little coterie of writers/editors/reviewers all enabling each other. Peer review is asking the question, are you really in my peer group? And they've finally succeeded in establishing their own fake peer group.
It's possible that the long timelines are part of this function as well. If an author is out of the profession a year later, maybe their paper isn't worth publishing. Impact is only for those who stick it out.
The one paper I published from my dissertation didn't come out for six years. I had already published my post-doc work and left research for teaching by that point. My child (born the year I graduated) was already in grade school.
This was only partially due to reviewers. It was much more the stresses of post-doc / early parenthood and the not-great writing dynamic between me and my grad school advisor.
Good luck with your experiment(s). I'm going to forward this to a blog-curious grad student who I just published in my newsletter.
Congratulations, you have another reader of your phd thesis! Not all of it, but I looked at the paper "Neuron-level Prediction and Noise can Implement Flexible Reward-Seeking Behavior". Mostly because I'm losing faith in SGD and backprop in connectionist AI.
I don't think substack is the best place for formal peer review, but maybe somebody will listen to me, so here's a very brief go.
I found that analyzing a local-update algorithm in a particular two neuron setting to be very interesting. I thought it was informative because in RL I was never a fan of explicitly engineering reward seeking, and I do prefer generative (model-based-ish) methods. In particular the noice component reminds me a lot of advancements of generative methods in deep learning (langevin dynamics).
I'm not fully convinced that predictive coding is the way to go, but the biggest unanswered questions in neural network science is continual learning and avoiding catastrophic forgetting: how can ANNs learn and *maintain* underlying structure of the world without resorting to methods like replay (combine inherited and learned underlying structure + long term memory for specifics).
So for your paper, focusing on the general bandit problem, the most interesting follow up would be how sample efficient transferring the network to a different bandit problem is. I didn't quite get that from the current paper. Furthermore, once you switch back to the original formulation of the bandit problem, can the network remember it? Would it become more sample efficient Etc. Can all this be done without extensive engineering like meta-reinforcement?
I enjoyed reading this paper, and while it's not groundbreaking and leaves a lot of unanswered questions, no paper is, I think you have the right ideas!
Substack might not be the best place for peer review, but who knows where that is at this point.
A lot of the questions you raise are ones I’m interested in, too. Re: predictive coding (PC) over other learning rules: I’ve been testing other local rules that have been successful in various tasks in the literature—Hebbian learning rules or covariance-based plasticity—and while these rules replicate performance in bandit tasks, there are advantages to PC because there’s a neat way to write feedforward neural activity updates as minimizing an energy function, and PC is the weight rule that minimizes the same energy function. This gives you a lot of nice properties other energy-based networks have. But I’m not married to PC either; I just think it’s the best candidate so far.
Interesting question about task transfer, too—it’s not something I’ve considered much. Now that you say it, though, I do have a figure draft for the next paper that I could modify to address that exact question, if I’ve understood you correctly (performance before vs. after switching between bandit tasks w/ different reward probabilities). I’ll make a note of it!
In task transfer, I would be more interested in sample and computational efficiency. A collapse of performance would be an unfortunate but interesting result
I thought about this a bit more... maybe another interesting formulation (and one more suited to the exploration of your paper) could be "quicker convergence to steady attractor dynamic state". Anyways these things may be harder to measure and look for because it's essentially about second order effects.
Intuitively with the energy based formulation, further samples should increase the steepness of the parameter gradient landscape per sample even when the problem is modified somewhat
Peer review is for academics, quality control is for everyone else. Peer reviews are about as useful as exams, testing a person's knowledge and insight at a given point in time. It's a given that people and experiments evolve over time. Of course, there are people who are happy to be academics, where peer-reviewed papers ARE their accomplishments. Applied science and engineering doesn't interest them, or is someone else's responsibility.
Maybe a curated ScienceStack would be more effective for ongoing collaboration than a stack comprised of readers without relevant expertise. GitHub is surely more efficient if you're writing code.
I would back a ScienceStack for sure--wonder if there's something out there; I think one of the PNAS papers mentioned a similar idea. If features included some background verification and an incentive toward readability I would be quite happy
Oh cool, thanks for sharing. I was only aware of eLife and the machine learning conferences that use OpenReview, but PLOS does mention a couple others in this post
And they do discuss the possibility of wider community engagement, which eLife/OpenReview don’t as far as I’m aware. It’ll be fun to follow where these go
I have SO much to say in this as a fellow scientist who has similar experience to your own. I agree with you in essence. But I don't want to through the baby out with the bathwater. Substack definitely won't be better imo 😂. What needs to change in academia is the incentive structure. When the primary academic capital is papers, “publish or perish”, then ppl are starting to do anything to get them. Compounded with the degradation of situation in paying thousands to have your work published, these academic publishers have a lot to answer for! If we shifted the academic KPI to a more well rounded set of incentives, team work, collaboration, support etc we can start to change behaviour as well.
While I doubt Substack would scale, it certainly has worked better for me so far! Christopher's comments on my paper, for example, were as or more thorough than peer reviews I've gotten, and _much_ faster. Plus, comments on my last two posts have really helped me understand what I take for granted when I'm explaining something, and what connections I need to make more concrete, including for myself.
Also, given the discussion in the PNAS paper, along with Adam's, mine, and my colleagues' experiences, I'm not sure there is any baby in all that bathwater anymore. Substack has its own misaligned incentives, certainly, but at least it's different from what's not working, e.g. the fake papers Phil mentions above.
And Wikipedia is flawed, but it's still such a remarkable thing! The spirit of this experiment is just trying things--good enough over perfection, and change over stagnancy. Why not?
I understand what you mean but changing the platform to one such as this will create new problems. You won't improve feedback and efficiency by opening academic papers to everyone. It is not feasible, and Substack has a lot of problematic ideas on here that use science as a cover, and many of them in space relevant to my work is repackaged eugenics. So I really don't want ideologically motivated feedback. In which case the baby is the peer review itself, but perhaps a new platform needs to be created to change it from the broken model we have now. The other point to note is obviously very good science is still being published that is also the baby here.
Your point about communication to the public or those outside of your field is simply what you are learning about sci comm, again the platform doesn't matter much for that.
Also yes wiki is improving and not perfect but that is my criticism of it, no toxic positivity required.
oh I want to add that Wikipedia still isn't rigorous either, I once tried to edit some science on there about a project I was directly working on and I got patronizingly rejected. They often lack evidence synthesis skills and nuance, although are getting better. 👍🏼
I think people underestimate how good feedback from non domain experts can be. They’re usually free from the kinds of assumptions everyone in your field already makes, meaning they will just ask the “dumb” questions. And personally, I’ve been surprised by how many supposedly dumb questions I struggle to answer, and they tend to further solidify my foundations by thinking through them.
Agree so much!
Here me out: move peer reviews to github
I’m confident it addresses literally every issue (pun intended) you list
I see the possibility of a beautiful friendship @Scott Lipscomb
Non-scientist, so completely unqualified here. But I have a theory: that lots of systems don't work the way you might think they do. Peer review looks like one of those: it looks like a system where other scientists make meaningful comments on your work. But in fact it's just a way of making sure that *someone* has read the paper, which prevents scientists from just writing nonsense.
There is a story in the Economist about how the number of fake papers is rising, and it points out that there's a little coterie of writers/editors/reviewers all enabling each other. Peer review is asking the question, are you really in my peer group? And they've finally succeeded in establishing their own fake peer group.
It's possible that the long timelines are part of this function as well. If an author is out of the profession a year later, maybe their paper isn't worth publishing. Impact is only for those who stick it out.
The one paper I published from my dissertation didn't come out for six years. I had already published my post-doc work and left research for teaching by that point. My child (born the year I graduated) was already in grade school.
This was only partially due to reviewers. It was much more the stresses of post-doc / early parenthood and the not-great writing dynamic between me and my grad school advisor.
Good luck with your experiment(s). I'm going to forward this to a blog-curious grad student who I just published in my newsletter.
https://randallhayes.substack.com/p/grey-matter-clear-mind
Congratulations, you have another reader of your phd thesis! Not all of it, but I looked at the paper "Neuron-level Prediction and Noise can Implement Flexible Reward-Seeking Behavior". Mostly because I'm losing faith in SGD and backprop in connectionist AI.
I don't think substack is the best place for formal peer review, but maybe somebody will listen to me, so here's a very brief go.
I found that analyzing a local-update algorithm in a particular two neuron setting to be very interesting. I thought it was informative because in RL I was never a fan of explicitly engineering reward seeking, and I do prefer generative (model-based-ish) methods. In particular the noice component reminds me a lot of advancements of generative methods in deep learning (langevin dynamics).
I'm not fully convinced that predictive coding is the way to go, but the biggest unanswered questions in neural network science is continual learning and avoiding catastrophic forgetting: how can ANNs learn and *maintain* underlying structure of the world without resorting to methods like replay (combine inherited and learned underlying structure + long term memory for specifics).
So for your paper, focusing on the general bandit problem, the most interesting follow up would be how sample efficient transferring the network to a different bandit problem is. I didn't quite get that from the current paper. Furthermore, once you switch back to the original formulation of the bandit problem, can the network remember it? Would it become more sample efficient Etc. Can all this be done without extensive engineering like meta-reinforcement?
I enjoyed reading this paper, and while it's not groundbreaking and leaves a lot of unanswered questions, no paper is, I think you have the right ideas!
Cool thanks for checking it out!
Substack might not be the best place for peer review, but who knows where that is at this point.
A lot of the questions you raise are ones I’m interested in, too. Re: predictive coding (PC) over other learning rules: I’ve been testing other local rules that have been successful in various tasks in the literature—Hebbian learning rules or covariance-based plasticity—and while these rules replicate performance in bandit tasks, there are advantages to PC because there’s a neat way to write feedforward neural activity updates as minimizing an energy function, and PC is the weight rule that minimizes the same energy function. This gives you a lot of nice properties other energy-based networks have. But I’m not married to PC either; I just think it’s the best candidate so far.
Interesting question about task transfer, too—it’s not something I’ve considered much. Now that you say it, though, I do have a figure draft for the next paper that I could modify to address that exact question, if I’ve understood you correctly (performance before vs. after switching between bandit tasks w/ different reward probabilities). I’ll make a note of it!
In task transfer, I would be more interested in sample and computational efficiency. A collapse of performance would be an unfortunate but interesting result
Isn't efficiency tied to performance, and how many trials it takes to achieve that performance?
I thought about this a bit more... maybe another interesting formulation (and one more suited to the exploration of your paper) could be "quicker convergence to steady attractor dynamic state". Anyways these things may be harder to measure and look for because it's essentially about second order effects.
Intuitively with the energy based formulation, further samples should increase the steepness of the parameter gradient landscape per sample even when the problem is modified somewhat
Peer review is for academics, quality control is for everyone else. Peer reviews are about as useful as exams, testing a person's knowledge and insight at a given point in time. It's a given that people and experiments evolve over time. Of course, there are people who are happy to be academics, where peer-reviewed papers ARE their accomplishments. Applied science and engineering doesn't interest them, or is someone else's responsibility.
Maybe a curated ScienceStack would be more effective for ongoing collaboration than a stack comprised of readers without relevant expertise. GitHub is surely more efficient if you're writing code.
I would back a ScienceStack for sure--wonder if there's something out there; I think one of the PNAS papers mentioned a similar idea. If features included some background verification and an incentive toward readability I would be quite happy
If such a venue can help with paying the bills, even better.
Would the open review at PLOS qualify? I've never submitted anything there so have no experience.
https://plos.org/open-science-practice/
Oh cool, thanks for sharing. I was only aware of eLife and the machine learning conferences that use OpenReview, but PLOS does mention a couple others in this post
https://plos.org/resource/open-peer-review/
And they do discuss the possibility of wider community engagement, which eLife/OpenReview don’t as far as I’m aware. It’ll be fun to follow where these go
I have SO much to say in this as a fellow scientist who has similar experience to your own. I agree with you in essence. But I don't want to through the baby out with the bathwater. Substack definitely won't be better imo 😂. What needs to change in academia is the incentive structure. When the primary academic capital is papers, “publish or perish”, then ppl are starting to do anything to get them. Compounded with the degradation of situation in paying thousands to have your work published, these academic publishers have a lot to answer for! If we shifted the academic KPI to a more well rounded set of incentives, team work, collaboration, support etc we can start to change behaviour as well.
While I doubt Substack would scale, it certainly has worked better for me so far! Christopher's comments on my paper, for example, were as or more thorough than peer reviews I've gotten, and _much_ faster. Plus, comments on my last two posts have really helped me understand what I take for granted when I'm explaining something, and what connections I need to make more concrete, including for myself.
Also, given the discussion in the PNAS paper, along with Adam's, mine, and my colleagues' experiences, I'm not sure there is any baby in all that bathwater anymore. Substack has its own misaligned incentives, certainly, but at least it's different from what's not working, e.g. the fake papers Phil mentions above.
And Wikipedia is flawed, but it's still such a remarkable thing! The spirit of this experiment is just trying things--good enough over perfection, and change over stagnancy. Why not?
I understand what you mean but changing the platform to one such as this will create new problems. You won't improve feedback and efficiency by opening academic papers to everyone. It is not feasible, and Substack has a lot of problematic ideas on here that use science as a cover, and many of them in space relevant to my work is repackaged eugenics. So I really don't want ideologically motivated feedback. In which case the baby is the peer review itself, but perhaps a new platform needs to be created to change it from the broken model we have now. The other point to note is obviously very good science is still being published that is also the baby here.
Your point about communication to the public or those outside of your field is simply what you are learning about sci comm, again the platform doesn't matter much for that.
Also yes wiki is improving and not perfect but that is my criticism of it, no toxic positivity required.
oh I want to add that Wikipedia still isn't rigorous either, I once tried to edit some science on there about a project I was directly working on and I got patronizingly rejected. They often lack evidence synthesis skills and nuance, although are getting better. 👍🏼
My feedback on this piece is that it desperately needs some puns.
Feedback noted 🫡
and please note that I won't be able to approve your next blog post unless you cite this comment
and include links to a series of my Amazon Affiliate links to various off-brand products of dubious quality
I really thought the experiment was going to survive longer than this
isn't a failed hypothesis still a success scientifically?