Jump to: navigation, search
MithiWiki Home > ConnectXf Home > ConnectXf Administration > Configuration > Error:"access beyond end of device"

Troubleshooting Icon.png
Product ConnectXf
Version All
Applies to Administrators
Level Advanced

I get "access beyond end of device" errors

Your kernel log fills with

attempt to access beyond end of device 
drbd0: rw=1, want=211992584, limit=211986944
Buffer I/O error on device drbd0, logical block 26499072

Your file system then remounts read-only, panics or similar. When you try to fsck, you get something like
The filesystem size (according to the superblock) is ... blocks.
The physical size of the device is ...+x blocks.
Envision this:
|-- usable area with drbd and internal meta data --|-IMD-|
|-- real device -----------------------------------------|
IMD is "internal meta data". Once created, it is fixed size. With drbd 0.7 it was fixed 128MB. With drbd 8.0 it is approximately [total storage of real device]/4/8/512/2 rounded up, +36k, rounded up to the next 4k.

grep -e hda4 -e drbd0 /proc/partitions
  3     4  105996744 hda4
147     0  105993472 drbd0
ceil(105996744 kB / 32768) == 3235 kB
                           +    36 kB
                           == 3271 kB
               4k  aligned == 3272 kB
    105996744 kB - 3272 kB == 105993472 kB

If you did mkfs /real/device, then later mount through DRBD, the file system either recognized size mismatch in superblock vs. actual block device size on the spot and refuse to mount (xfs does this, iirc).
Or the file system mounts alright, because it skips the check for block device size (ext3, at least certain version of it, aparently do this; it is ok for a file system to assume that its superblock contains valid data) and then thinks it could use the now not available space which is occupied by IMD.
There are various ways to find out what your file system thinks about the usable space it occupies. For ext3, you can find out with e.g.

 tune2fs -l /dev/whatever | 
 awk '/^Block.size:/ { bs=$NF }
      /^Block.count:/ { bc=$NF }
      END { print bc * bs / 1024, "kB" }'

As long as the file system does not want to use that area, it won't notice. If the file system eventually decides to use that area, whops, surprise, it gets an access beyond end of device error. When the file system will start using that area is nearly impossible to pretict. So it may appear to work fine for month, and then suddenly break again and again.