Tests are supposed to help keep apps from being flaky. But once in a while, tests themselves can become flaky—even the most straightforward ones. 下面是我们如何深入到一个有问题的测试 Ruby on Rails PostgreSQL支持的应用程序，以及我们发现的.

We wanted to check that certain business logic (called by a method perform)不会改变 calendar 模型(的实例) Calendar, a Ruby on Rails ActiveRecord model class) so we wrote:

Let (:calendar) {create(:calendar)}
specify do
  expect do
    执行# call业务操作
    calendar.reload
  end
    .Not_to change(日历，:attributes)
end

This was passing in one development environment (MacOS), 但它在CI (Linux)中几乎总是失败。.

Fortunately, we managed to reproduce it on another development environment (Linux), 它失败的地方是:

expected `Calendar#attributes` not to have changed, but did change from {"calendar_auth_id"=>8,
"created_at"=>2020-01-02 13:36:22.459149334 +0000, "enabled"=>false, "events_...t_sync_token"=>nil,
"title"=>nil, "updated_at"=>2020-01-02 13:36:22.459149334 +0000, "user_id"=>100} to {
"calendar_auth_id"=>8, "created_at"=>2020-01-02 13:36:22.459149000 +0000, "enabled"=>false,
"events_...t_sync_token"=>nil, "title"=>nil, "updated_at"=>2020-01-02 13:36:22.459149000 +0000, "user_id"=>100}

看到可疑的东西了吗??

调查

仔细检查后，我们发现 created_at and updated_at 的内部对时间戳进行了轻微更改 expect block:

{"created_at"=>2020-01-02 13:36:22.459149334 +0000, "updated_at"=>2020-01-02 13:36:22.459149334 +0000}
{"created_at"=>2020-01-02 13:36:22.459149000 +0000, "updated_at"=>2020-01-02 13:36:22.459149000 +0000}

秒的小数部分被截断，这样 13:36:22.459149334 became 13:36:22.459149000.

我们确信 perform 没有更新 calendar object, so we formed a hypothesis that the timestamps were being truncated by the database. To test this, we used the most advanced debugging technique known, i.e., puts debugging:

Let (:calendar) {create(:calendar)}
specify do
  expect do
    在perform: #{calendar之前放".created_at.to_f}"
    perform
    在perform: #{calendar后面放".created_at.to_f}"
    calendar.reload
    重新加载后:#{日历.created_at.to_f}"
  end
    .Not_to change(日历，:attributes)
end

但是在输出中看不到截断:

执行前:1577983568.550754
执行后:1577983568.550754
重载后:1577983568.550754

这是相当令人惊讶的访问器 #created_at should have had the same value as the attribute hash value of 属性(“created_at”). To be sure we were printing the same value used in the assertion, we changed the way we accessed created_at.

而不是使用访问器 calendar.created_at.to_f, we switched to fetching it from the attributes hash directly: calendar.属性(“created_at”).to_f. 我们对 calendar.reload were confirmed!

执行前:1577985089.0547702
执行后:1577985089.0547702
重载后:1577985089.05477

如你所见，打电话 perform didn’t change created_at, but reload did.

To ensure the change was not happening on another instance of calendar and then being saved, we performed one more experiment. We reloaded calendar 考试前:

让(:calendar){创建(:calendar).reload }
specify do
  expect do
    perform
    calendar.reload
  end
    .Not_to change(日历，:attributes)
end

这使得测试结果是绿色的.

The Fix

Knowing that it’s the database that truncates our timestamps and fails our tests, we decided to prevent the truncation from happening. We generated a DateTime 对象，并将其舍入为整秒. Then, we used this object to set Rails’ Active Record timestamps explicitly. 此更改修复并稳定了测试:

let(:time) { 1.day.ago.round }
let(:calendar) { create(:calendar, created_at: time, updated_at: time) }

specify do
  expect do
    perform
    calendar.reload
  end
    .Not_to change(日历，:attributes)
end

The Cause

为什么会发生这种情况?? 活动记录时间戳为 由Rails设置的 ActiveRecord::时间戳 module using Time.now. Time precision is OS-dependent如文件所述 可能包括小数秒.

We tested Time.now resolution on MacOS and Linux with a script that counts frequencies of fractional part lengths:

pry> 10000.times.map { Time.now.to_f.to_s.match(/\.(\d+)/)[1].size }.group_by{|a| a}.映射{|k, v| [k, v.count]}.to_h

# MacOS => {6=>6581, 7=>2682, 5=>662, 4=>67, 3=>7, 2=>1}
# Linux => {6=>2399, 7=>7300, 5=>266, 4=>32, 3=>3}

As you can see, about 70% of timestamps on Linux had seven digits of precision after the decimal, 而在MacOS上只有25%. This is the reason the tests passed most of the time on MacOS and failed most of the time on Linux. You may have noticed that the test output had nine-digit precision—that’s because RSpec uses Time#nsec 格式化时间输出.

当Rails模型保存到数据库时, any timestamps they have are stored using a type in PostgreSQL called 没有时区的时间戳, which has 微秒的决议—i.e.，小数点后六位. So when 1577987974.6472975 is sent to PostgreSQL, it truncates the last digit of the fractional part and instead saves 1577987974.647297.

The Questions

这其中的原因还是个问题 calendar.created_at 在我们调用时没有重新加载 calendar.reload, even though calendar.属性(“created_at”) was reloaded.

同样，结果 Time 精度测试有点令人惊讶. We were expecting that on MacOS, the maximal precision is six. 我们不知道为什么 sometimes 有七位数. What surprised us more was the distribution of the value of the last digits:

pry> 10000.times.map { Time.now}.map{|t| t.to_f.to_s.match(/\.(\d+)/)[1] }.select{|s| s.size == 7}.group_by {| | e e [1]}.映射{|k, v| [k, v.size]}.to_h

# MacOS => {"9"=>536, "1"=>555, "2"=>778, "8"=>807}
# Linux => {"5"=>981, "1"=>311, "3"=>1039, "9"=>309, "8"=>989, "6"=>1031, "2"=>979, "7"=>966, "4"=>978}

As you can see, the seventh digit on MacOS is always 1, 2, 8, or 9.

If you know the answer to either of these questions, please share an explanation with us.

The Future

事实上，Ruby on Rails Active Record timestamps are generated on the application side may also hurt when those timestamps are used for reliable and precise ordering of events saved to the database. As application server clocks may be desynchronised, events ordered by created_at may appear in a different order than they really occurred. To get 更可靠的行为, it would be better to let the database server handle timestamps (e.g., PostgreSQL’s now()).

然而，这是一个值得另写一篇文章的故事.

---

特别感谢 Gabriele Renzi 感谢您帮助创建这篇文章.